View PDF (https://arxiv.org/pdf/2607.08374)HTML (experimental) (https://arxiv.org/html/2607.08374v1) Abstract:Personality recognition has traditionally been constrained by theory-dependent formulations, where models are trained to fit predefined psychological taxonomies rather than uncovering shared underlying behavioral structure. This limits generalization, as personality itself is better understood as theory-invariant, while existing annotations reflect only partial and sometimes inconsistent views of the same latent traits. In this work, we introduce JAM ((J)udge for (A)daptive (M)etric-Alignment), a theory-agnostic framework that shifts learning from adapting to predefined personality theories toward discovering unified latent pseudo-facets that capture shared psychological structure. Rather than constraining the model to any personality taxonomy during training or inference, the framework learns generalizable psychological representations and can infer an individual's latent psychological profile directly from the textual samples, without requiring theory-specific labels. JAM achieves this through an Attention-Pooled Graph Prototypical Network that learns structured representations via clustering in embedding space, together with a Cross-Theory Harmonization (CTH) approach that integrates (i) Human-Guided Linkage and (ii) Machine-Induced Consensus to unify heterogeneous datasets without relying on predefined labels. To further improve robustness and data quality, we incorporate an LLM-as-a-Judge mechanism operating in two configurations, (i) LLM-before-the-loop and (ii) LLM-in-the-loop which identifies ambiguous samples to guide adaptive metric learning. Experiments show that JAM improves cross-framework generalization and performance, establishing a strong step toward theory-agnostic personality inference and supporting low-resource personality theories. The related code repository, model weights, and artifacts are available at this https URL (https://research.jingjietan.com/JAM) Subjects:Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Robotics (cs.RO); Social and Information Networks (cs.SI) Cite as:arXiv:2607.08374 (https://arxiv.org/abs/2607.08374) [cs.CL] (or arXiv:2607.08374v1 (https://arxiv.org/abs/2607.08374v1) [cs.CL] for this version) https://doi.org/10.48550/arXiv.2607.08374 arXiv-issued DOI via DataCite (pending registration) Journal reference:IEEE Transactions on Affective Computing (2026) Related DOI:https://doi.org/10.1109/TAFFC.2026.3712379 DOI(s) linking to related resources Submission history From: Jing Jie Tan [view email (https://arxiv.org/show-email/7c953d40/2607.08374)] • *[v1]** Thu, 9 Jul 2026 11:49:14 UTC (5,585 KB)