Project 7: Attention to symptom keywords¶
Mentors: To be announced
Problem: A model may land on the right diagnosis-adjacent word for the wrong reason, and without inspecting attention there's no way to tell whether it actually looked at the symptom words a clinician would flag.
Context: Built on Symptom2Disease (Apache-2.0, Hugging Face) input_text patient messages describing symptoms — a standalone message is enough here since the analysis is per-message, not turn-to-turn (unlike Project 3, which needs the multi-turn MTS-Dialog set), anchored to ML4LLM Ch.6 · proj35: Raw and softmax attention scores (helper).
Goals: When the model generates a diagnosis-adjacent token, does attention actually concentrate on the symptom words a clinician would flag as relevant?
Deliverables: A notebook that extracts raw and softmax attention scores for the token generated after a symptom description, overlays attention weight on the input tokens, and checks whether the highest-attention tokens correspond to clinically salient symptom words versus filler text.
Showcase: TBD
References:
- Attention is not Explanation (Jain & Wallace, NAACL 2019): the foundational, and contested, result this project's core question tests on medical text, whether attention weights actually track the tokens that matter to a prediction.