Absenteeism at Work: ML

Below is a compact, research-grounded reference list plus actionable guidance: why absenteeism happens (what studies find), which predictors are strongest, concrete feature-engineering ideas, and how to model/classify absence reasons with ML (supervised + unsupervised). I can expand any section (code, example features, or a pipeline) if you want.

1. Key recognized studies & datasets (quick pointers)

• Farrell & Stamm — Meta‑Analysis of the Correlates of Employee Absence (1988): shows work‑environment and organization‑level factors are stronger predictors than demographics or pure psychological traits.
• Systematic review — Čikeš et al., The Determinants and Outcomes of Absence Behavior (2018): summarizes ~100 empirical studies; highlights attitudes (job satisfaction, commitment), personality, health/stress, family demands, and organizational policies as frequent determinants.
• Harrison & Martocchio — 20‑Year review of absenteeism research (covers trends, measures, and theory). Useful for measurement issues (frequency vs. duration).
• ML papers & applications — examples applying ML to predict absence (e.g., “Predicting Employee Absence from Historical Absence Profiles with Machine Learning” — feature engineering + time‑lag aggregates; and several IEEE conference papers on absenteeism prediction). Good sources for practical ML methods and evaluation.
• Public dataset — UCI “Absenteeism at work” (Brazilian company dataset with reason codes + demographics + work items). Good for prototyping models and benchmarking feature ideas.

2. Reasons for absenteeism reported across studies (summary)

• Illness & medical reasons (acute sickness, chronic conditions) — commonly reported and often coded by ICD/reason categories in HR systems.
• Family/household obligations (childcare, eldercare, family emergencies).
• Work‑related factors: low job satisfaction, poor working conditions, high workload, low autonomy, shift patterns—these often increase voluntary absence. Organization‑level policies (absence controls, leave policies) also matter.
• Stress and mental health (burnout, depression, anxiety) — increasingly recognized as a major driver of both short‑ and long‑term absence.
• Attendance history & culture (previous absence patterns, team norms, disciplinary environment) — strong statistical predictor and often mediates other effects.
• Demographics & lifestyle (age, family size, smoking, BMI) — correlated but usually weaker predictors than work‑environment and history.

3. Best predictors (what to prioritize in models)

• Absence history features (recency, frequency, duration, time since last absence) — consistently among the strongest predictors in reviews and applied ML studies.
• Work environment / job design variables (job satisfaction, autonomy, shift type, workload, role clarity).
• Organizational policy & culture variables (absence control policies, HR practices, sector/firm size).
• Health & stress indicators (self‑reported stress, chronic conditions, mental‑health flags).
• Demographics and commute/lifestyle (age, distance to work, smoking/drinking, BMI) — add value but rank after history and work factors.

4. Feature engineering — concrete features to build (practical list)
   Numbered steps you can implement in a pipeline:

1. Build absence‑history lags and aggregates: count of absences in last 7/30/90/365 days; total absence hours in those windows; mean/median absence duration; time since last absence. These are essential predictors.
2. Encode reason codes into grouped categories (e.g., acute medical, chronic/long‑term, family, administrative). For free‑text reasons use NLP-topic modeling / simple keyword mapping.
3. Create seasonality and calendar features: month, week‑of‑year, day‑of‑week flags, public holiday proximity, school holiday overlap (affects family absence).
4. Interaction and ratio features: workload per hour × recent absence; age × tenure; commute distance / transport expense. Interactions can expose stressors.
5. Health‑score composites: combine BMI, smoking, drinking, chronic conditions into a single health risk index (standardize first).
6. Organizational/contextual features: team absence rate (peer influence), manager absence, policy severity index (points for disciplinary failures, warnings).
7. Rolling statistical features for time‑series models: moving averages, exponentially weighted means of absence hours, seasonal decomposition residuals for anomalies.
8. Behavioral features from access/IT systems (if available): late arrivals, partial day leaves, login patterns — useful proxies for presenteeism/engagement. (See review of measurement issues.)

5. How to classify/cluster reasons with ML — practical approaches

• Supervised classification (when you have labeled reason categories)

  1. Preprocess: map raw reason codes to a manageable set (e.g., 5–8 classes: health, mental health, family, work-related, administrative, other). Use one‑hot or ordinal encoding as appropriate.
  2. Models to try: Random Forest / XGBoost / LightGBM (fast baseline + feature importance), multinomial logistic regression (interpretable), and neural networks if you have lots of data or text fields. Ensemble methods tend to perform well on tabular HR data. Evaluate with precision/recall per class, macro F1, and a confusion matrix (class imbalance expected).
  3. For multi‑label reasons (multiple causes per absence) use binary relevance, classifier chains, or multi‑label neural nets and evaluate with subset accuracy and Hamming loss.

• Unsupervised & semi‑supervised methods (when labels are absent or noisy)

  1. Cluster absence episodes using engineered features (history, duration, time, health proxies). Algorithms: k‑means or GMM for globular clusters; DBSCAN/HDBSCAN for irregular shapes; hierarchical clustering to produce a taxonomy. Use PCA/UMAP for visualization and cluster validation (silhouette, Davies‑Bouldin).
  2. Use topic modeling (LDA) or sentence embeddings + clustering if you have free‑text reason notes — this often reveals latent reason groups (e.g., “family emergency”, “flu”, “doctor visit”).
  3. Combine clustering with supervised labels in a semi‑supervised loop: cluster to propose labels, validate with a subset of human‑annotated episodes, then train a classifier. This works well when reasons are inconsistently coded.

• Explainability & fairness

  • Use SHAP/ICE plots or feature importance to explain predictions to HR stakeholders. Check for bias by subgroup (age, gender, race) and avoid decisions that could discriminate.

6. Evaluation & operational considerations

• Choose the target carefully: absence frequency, total time lost, or episode reason—each has different drivers and different modeling choices. Measurement differences are important (frequency vs duration) per the reviews.
• Use time‑aware validation (temporal cross‑validation) to avoid leakage from future to past. Test models on held‑out time periods.
• Keep workers’ privacy and legal constraints in mind: health data often needs consent and special handling.

If you want, I can next:

1. produce a checklist of engineered features as a downloadable CSV template for your dataset, or
2. build a starter notebook (using the UCI dataset) that implements feature engineering + a baseline classifier and SHAP explanations. Which would you prefer?