neural chat

**Neural Chat** is a **7B parameter language model developed by Intel as a fine-tune of Mistral-7B, aligned using Direct Preference Optimization (DPO) and optimized to showcase high-performance LLM inference on Intel hardware** — demonstrating that competitive language models can run efficiently on Intel Gaudi2 accelerators and Intel Xeon CPUs without requiring NVIDIA GPUs, using the Intel Extension for Transformers (ITREX) for advanced INT8/INT4 quantization. **What Is Neural Chat?** - **Definition**: A fine-tuned language model from Intel Labs — starting from Mistral-7B base, further trained with supervised fine-tuning on high-quality instruction data (OpenOrca), then aligned using DPO (Direct Preference Optimization) to improve response quality and helpfulness. - **Intel Hardware Showcase**: Neural Chat is designed to demonstrate that high-quality LLM inference doesn't require NVIDIA GPUs — Intel optimized the model to run efficiently on Intel Gaudi2 AI accelerators, Intel Xeon Scalable processors, and Intel Arc GPUs. - **Leaderboard Achievement**: At release, Neural Chat V3.1 topped the Hugging Face Open LLM Leaderboard for the 7B parameter category — beating the base Mistral-7B model and demonstrating the value of DPO alignment. - **ITREX Optimization**: The Intel Extension for Transformers provides advanced quantization (INT8, INT4, mixed precision) and kernel optimizations specifically for Intel hardware — enabling Neural Chat to run at competitive speeds on CPUs that are typically considered too slow for LLM inference. **Key Features** - **DPO Alignment**: Uses Direct Preference Optimization rather than RLHF — a simpler alignment method that directly optimizes the model from preference pairs without training a separate reward model. - **CPU-Optimized Inference**: Intel's optimizations make Neural Chat one of the fastest models to run on x86 CPUs — important for enterprise deployments where GPU availability is limited. - **INT4 Quantization**: ITREX provides INT4 quantization with minimal accuracy loss — reducing memory requirements by 8× and enabling inference on standard server CPUs. - **OpenVINO Integration**: Neural Chat can be exported to OpenVINO format for optimized inference on Intel hardware — including Intel integrated GPUs and Intel Neural Processing Units (NPUs) in laptops. **Neural Chat is Intel's demonstration that competitive LLM performance doesn't require NVIDIA hardware** — by fine-tuning Mistral-7B with DPO alignment and optimizing inference with ITREX quantization, Intel proved that high-quality language models can run efficiently on Xeon CPUs and Gaudi accelerators, expanding the hardware options for enterprise AI deployment.