Multimodal: Hierarchical Transformer-based Multi-task Learning

Multimodal for a research project based on stock price moments using CEOs audios and transcripts to analyse how companies are doing financially using Hierarchial Transformers

Dataset: Earnings Call Data, CEOs Audio Data & Transcripts

Transformer: XLSR-Wav2Vec2, BERT, Hierarchial Transformers for Single task, Multi task

Stock Price Prediction/Forecasting: This is about predicting the future price of stocks based on historical data and other relevant information.

It was a multimodal made using multiple data types involving financial data, audio & text data.

Preprocessing of Audio data has also been done using GPU

4 big models were made for evaluation- single task, multi-task, text data and text+audio data.

Reproduction of a research paper

Hierarchical level layers used in building transformers where the output from previous layers used as input for next layer and each layer does different tasks.

Heavy audio data caused multiple crashes.

It was handled with reduced data and GPU preprocessing and training.