Multi-party chat platforms used in technical support and developer collaboration often contain overlapping, interleaved conversations that are difficult to follow and analyze. This paper revisits the task of thread disentanglement with a lightweight, two-stage pipeline that first predicts reply-to links using fine-tuned large language models (LLMs), then reconstructs threads using graph-based clustering methods. When applied zero-shot to out-of-domain Discord developer chats our approach demonstrates strong performance, achieving 0.90 Shen F-score and 0.87 one-to-one accuracy. We also show that combining models with complementary behaviors through ensemble voting improves robustness and generalization. Our findings highlight the potential of decoder-based LLMs, even in low-resource fine-tuning settings, to outperform traditional feature-engineered approaches and enable accurate thread disentanglement across platforms. This capability supports downstream applications such as developer assistance, chat summarization, and conversational search.