陆奕衡 Yiheng Lu 陆奕衡 Yiheng Lu

NVSpeech is an integrated pipeline for modeling human-like speech with paralinguistic vocalizations, bridging dataset construction, paralinguistic-aware ASR, and controllable TTS in a unified framework. In this project, I contributed to the recognition and synthesis side by developing timestamp-based paralanguage and prosody feature extraction with WhisperX, FunAudio, and Whisper-timestamped, evaluating robustness across Emilia and EARS MM_TTS, generating over 100 text slices for each paralanguage label to support synthesis and evaluation, and designing the project homepage to present the system architecture, demos, and datasets.

SchenkerHnet is my independent research project on structure-aware symbolic music generation, which aims to bring Schenkerian hierarchy into AI models so that generated music reflects long-range tonal organization rather than only local token patterns. To build this system, I reconstructed large-scale classical piano corpora into theory-guided training data with harmonic, tonal, notes density, and structural annotations, redesigned the representation from coarse beat-level tokens to a cleaner tick-level JSONL format, and developed a hierarchical encoder–router–downsampler–main network–upsampler pipeline with Schenker-aware routing and long-memory modeling inspired by dynamic chunking network (Hnet) and Mamba. The project is currently being scaled to larger corpora for interpretable and structurally coherent long-form music generation.

This project studies long-term electricity price forecasting through a hybrid framework that combines structured trend modeling with event-aware reasoning over news, aiming to capture both stable market fundamentals and high-impact external shocks. I contributed to the event-data and forecasting pipeline by building time- and keyword-conditioned news retrieval workflows for GDELT, automating the conversion from LLM-generated search plans to executable queries, collecting official yearly price-driver data and forecasts from EIA-oriented sources, improving retrieval robustness against malformed outputs, rate limits, and overlong queries, and supporting later benchmark experiments that connect event-aware news signals with long-horizon forecasting evaluation.

HyCoSeg is a hybrid pipeline that integrates classical image processing with deep learning for COVID-19 lung lesion segmentation in CT scans. The system first employs a 2D U-Net (LungMask) to delineate lung regions, providing anatomical constraints for subsequent detection. Ground-glass opacities are segmented using Hounsfield Unit thresholding combined with morphological post-processing, while consolidation lesions are identified through a coarse-to-fine deep model enhanced with Gabor texture filtering. Pleural effusion is extracted via boundary-aware density thresholding followed by connected-component analysis.

I designed the overall technical framework, led data collection and preprocessing, fine-tuned the 2D U-Net model, and built the unified pipeline for three-type lesion recognition.

FingerSense is an AI system I independently developed for real-time analysis of piano hand movements. It takes an overhead piano-practice video as input and splits it into temporally aligned audio and video streams. The system leverages PianoMotion10M to generate a reference hand-motion sequence in MANO space from the audio, while reconstructing the observed hand-motion sequence from video using HaMeR. It then aligns the two motion streams, computes frame-level losses, and feeds the structured evidence into an LLM to produce professional, second-level precise feedback for performance correction.

We developed PaperHelper, a system designed to assist novice researchers in academic paper retrieval and management, addressing the rigidity and ambiguity of traditional RAG-based search systems. The system leverages a fine-tuned BART model to classify user queries and employs an MCP-based retrieval mechanism to traverse and interpret the entire MongoDB-backed corpus, enabling precise identification and aggregation of relevant literature.

As the team lead, I proposed the overall system architecture and coordinated task allocation. My primary contribution focused on backend development, including fine-tuning the BART model, designing and implementing MCP tools for structured retrieval, and integrating the frontend, large language models (LLMs), and database infrastructure into a unified, end-to-end system.