Machine Learning and Me

My name is Xiangyu, and I am a machine learning enthusiast. I started studying machine learning since 2016, and started to seriously engage in this topic since 2018.

Please visit my LinkedIn Page for contact information.

Here are some of the projects I have done. Please click on the links to go to detailed project pages.

  • Image Captioning: a project that uses RNN and CNN models to generate novel captions for an image.
    • Unlike traditional image captioning techinques, we use a model where the RNN and CNN are concatenated together to form a language model. The output of this language model is fed into another netword which generates the next word in the caption.
    • Word Embedding: GloVe embedding is used for word representation
    • Transfer Learning: VGG16 is used to generate high-level image features and is concatenated with RNN output
    • Caption Generation: Beam Search is used to generate high probability outputs
    • Metric: BLEU and ROUGE scores are used to measure performance
  • Automatic Colorization: a project that uses CNN and autoencoders to colorize a gray-scale image.
    • We use an autoencoder to colorize a black-and-white phote. The novel aspect is that high level features of the image is extracted from a previously trained deep CNN and injected into the input of the decoder.
    • Color Spaces: CIE LAB color space is chosen for its better approximation of human vision
    • Transfer Learning: VGG16 is used to extract high level features which are concatenated with mid-level features extracted by the encoder
    • Model Merging: VGG16 and encoder is merged before feeding into decoder
  • Tweet Classification using BERT: a project that uses the BERT word representation to identify useful tweets and sort them into different categories during a natural disaster or a mass casualty event.
    • Pytorch Model: Use bidirectional LSTM stack on top of a BERT representation to create the classification model
    • Baseline: A bidirectional LSTM using GloVE Twitter embedding as input and a fine-tuned BERT sequence classification model serves as baselines for comparison
    • Training: Model is trained with various "Patience" and learning rate decay schedules.
    • Metric: Model is compared using Matthews Correlation Coefficient
  • Emotion Detection: a tutorial on using transfer learning to detect facial emotions in an image.
    • We illustrate the use of transfer learning to use previously trained CNN on new applications.
  • Feature Engineering: a tutorial introducing feature engineering and data exploration
    • We illustrate the most basic and the most important part of machine learning, data exploration and feature engineering.