Quickstart#

emonet#

A library to model negative emotion severity in patients with adverse childhood events (ACE).

Contributors: @chris-santiago, @gonzalezeric, @costargc

Installation & Environment Setup#

Using Docker#

  1. Open Terminal on Mac or PowerShell on Windows from the root directory.

  2. When the application is open, in the command line, build the image using the following: docker build -t cjsantiago/emonet-model -f docker/model/Dockerfile .

  3. Run container docker run -it --name emonet -p 8888:8888 -v "${HOME}"/emonet-data:/home/jovyan/emonet-data  cjsantiago/emonet-model

    • Important: Training the model or running batch inference presumes that you have a emonet-data directory within your home folder, containing the original voice_labeling_report directory. This will allow you to replicate all batch preprocessing done prior to model training.

    • You can score file(s) or signal(s), either on their own or with your own custom DataLoader, without the data directory (described above).

    • See docker/model/README.md for more.

  4. Once the container has been created, you may access the files using one of the URLs generated in the CLI.

Using Conda#

  1. Clone this repo, then cd emonet

  2. Create a virtual environment

    • For training and notebooks, use conda env create -f env-base-plus.yml

    • For scoring, only, use conda env create -f env-base-yml

  3. Install emonet package, pip install -e .

NOTE: We’re installing in editable mode (-e flag) as we expect to run training and/or scoring from this cloned repo. Editable mode will symlink source code from the cloned repo directory to the appropriate Python interpreter, enabling source code edits and easy-access to our saved models under the saved_models directory.

Installing ffmpeg#

ffmpeg is required to convert .m4a to .wav. On Mac this can be installed via Homebrew. Skip this if you’re running via Docker.

Data Setup#

To use our original datset splits, we recommend downloading directly from our S3 bucket. This also removes the need to complete some time-consuming preprocessing steps.

Download from S3#

Assumes that you have the AWS CLI tool installed on your machine (and that you have our credentials :grinning:).

Within your home folder, create a directory called emonet-data. You could also use our directory setup script python emonet/dir_setup.py.

From the emonet-data directory, run this command to sync (copy) the required directories and files directly from our S3 bucket.

Note that this assumes our credentials are located within ~/.aws/credentials

aws s3 sync s3://gatech-emonet/eval/ .

From Scratch#

Once you’ve setup the environment and installed the emonet package:

Run python emonet/data_setup.py

Note: you can pass an optional number of max_workers to this command; the default is 8 (threads).

python emonet/data_setup.py 16

This script will run and perform the following:

  1. dir_setup.py: Set up a data directory within the home folder

  2. m4a_to_wav.py: Convert any .m4a files to .wav

  3. batch_resample.py: Resample files to 16,000Hz

  4. batch_vad.py: Run voice activity detection (VAD)

  5. data_prep.py: Create train/valid/test splits and respective manifests

  6. wav_splitter.py: Split .wav files into 8-second chunks, the create new train/valid/test manifests that use the chunked .wav files

Training#

Now that files have all been converted to WAV, preprocessed with VAD and split training, validation and testing sets, and chunked into 8-second segments:

Command Line Tool#

The easiest way to run the model is via the CLI:

Run

python emonet/train.py <num_workers> <max_epochs> <emotion>

and pass in the desired number of workers, epochs and emotion.

Example:

python train.py 12 300 anger

Python#

You can also train the model in Python:

Open a .py file or notebook and run

from emonet import DATA_DIR
from emonet.train import main

main(workers=12, epochs=300, emotion="sadness", use_wandb=False, data_dir=DATA_DIR)

and pass in the desired number of workers, epochs and emotion; you can log runs to Weights & Biases by setting use_wandb=True, and change the default data directory using the data_dir parameter.

Pretrained Models#

No pretrained models are included in this public-facing repo.

Scoring#

Command Line Tool#

The easiest way to score (using our pretrained models) is via our CLI tool, emonet. The syntax for this tool is:

emonet <emotion> <file to score>

Example:

emonet anger /Users/christophersantiago/emonet-data/wavs/6529_53113_1602547200.wav

Note: This CLI tool will run VAD on the .wav file, and can accept arbitrary length– despite model being trained on 8-second chunks. Therefore, you should use an original .wav of the sample you wish to score, not a .wav that’s been preprocessed with VAD.

Python#

You can also score via Python:

from emonet import DATA_DIR, ROOT
from emonet.model import EmotionRegressor

def get_saved(emotion: str):
    return ROOT.joinpath('saved_models', f'{emotion}.ckpt')


emotion = 'fear'
model = EmotionRegressor.load_from_checkpoint(get_saved(emotion))

file = 'path-to-my-file'

model.score_file(file=file, sample_rate=16000, vad=True)

See inference.py for an example of how we scored our testing sets.