Documentation

Project Overview

This project builds and evaluates an NLP pipeline for detecting loneliness self-disclosure in Reddit posts using the FIG-Loneliness dataset. Loneliness self-disclosure is often subtle and indirect, expressed through linguistic markers of social disconnection and relational dissatisfaction rather than direct statements. The pipeline compares five text representations and three classical classifiers, plus a fine-tuned DistilBERT model.

Dataset

FIG-Loneliness contains 5,633 Reddit posts annotated for loneliness self-disclosure by trained undergraduate research assistants and MTurk workers. Posts were collected from four subreddits (r/loneliness, r/lonely, r/youngadults, r/college) between 2018–2020 using the Pushshift API.

Preprocessing

All posts pass through a reproducible 8-step preprocessing pipeline before feature extraction. Preprocessing choices are kept minimal to avoid removing loneliness-relevant linguistic cues.

# Core preprocessing steps
text = ftfy.fix_text(raw_text)                  # 1. Unicode normalisation
text = unicodedata.normalize("NFKC", text)
text = bleach.clean(text, tags=[], strip=True)  # 2. HTML stripping
text = URL_PATTERN.sub(" ", text)               # 3. URL removal
text = MENTION_PATTERN.sub(" ", text)           # 3. Reddit mention removal
text = emoji.replace_emoji(text, " ")           # 3. Emoji removal
text = text.lower()                             # 4. Lowercasing
text = REPEATED_PATTERN.sub(r"\1\1", text)   # 4. Char normalisation
doc = nlp(text)                                 # 5. spaCy tokenisation
tokens_no_stop = [                              # 6. Stopword removal
    t.lemma_ for t in doc
    if not t.is_stop or t.text in NEGATIONS     # preserve negations!
]

Text Representations

Models & Training

Classical classifiers (LR, SVM, RF) were trained on each of the 5 non-neural representations, yielding 15 classical experiments. DistilBERT was fine-tuned end-to-end for 3 epochs with early stopping (patience=2) on validation F1.

# DistilBERT fine-tuning
training_args = TrainingArguments(
    eval_strategy="epoch",
    save_strategy="epoch",
    num_train_epochs=3,
    learning_rate=2e-5,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model="f1",
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized["train"],
    eval_dataset=tokenized["validation"],
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(patience=2)],
)

Evaluation

All models are evaluated using Precision, Recall, F1-Score, Accuracy, and ROC-AUC. The binary task targets the Lonely class (label=1). The optimal classification threshold is derived from the Youden J statistic (maximises TPR − FPR on the validation ROC curve) and applied at inference.

API Reference

The FastAPI backend is a read-only results server. All endpoints return pre-computed results from the pipeline. The API base URL is https://codenameberyl-fig-lone.hf.space/docs.

Deployment

The pipeline must run once offline (on a machine with GPU access for DistilBERT). Results are then uploaded to HuggingFace Spaces which serves the FastAPI backend. This Next.js app connects to that API.

# 1. Clone the Space repository
!git clone https://huggingface.co/spaces/your-username/fig-loneliness
%cd fig-loneliness

# 2. Install dependencies
!pip install -r requirements.txt

# 3. Run the full pipeline (requires T4 GPU for DistilBERT)
!python run_pipeline.py

# 4. Commit results back to the Space
!git add results/
!git commit -m "Add pipeline results"
!git push

Limitations

• —English-only: all models and features are trained on English Reddit text
• —Reddit-specific: vocabulary, style, and discourse patterns may not generalise to other platforms
• —Temporal: data covers 2018–2020 only; language patterns may have shifted
• —Construct overlap: loneliness shares linguistic features with depression and anxiety
• —No demographic data: fairness analysis across subgroups is not possible
• —GPU dependency: DistilBERT fine-tuning requires a GPU; CPU inference is very slow
• —Annotation noise: some posts have ambiguous loneliness status (majority vote used)

Citation

Dataset licence: CC BY-NC 4.0