Inkling – A Multimodal Foundation Model from Thinking Machines Lab

1. What is Inkling

Inkling is an open-weight multimodal foundation model developed by Thinking Machines Lab, utilizing a Mixture-of-Experts (MoE) architecture and natively supporting unified reasoning across text, image, and audio modalities with 41 billion active parameters. Rather than focusing on leading performance on a single benchmark, Inkling emphasizes balanced capabilities across a wide range of tasks, including agents, programming, vision, and audio. It is positioned as a backend inference engine that can be embedded into various workflows. Through the Tinker platform, developers can download weights and perform on-demand fine-tuning, enabling flexible deployment from research to production.

Image source: Official article
Image source: official article

Technical positioning and domain: Inkling belongs to the field of multimodal foundation models, focusing on general-purpose agents and tool calling scenarios. It employs a MoE sparse architecture with a total of 975 billion parameters, activating 6 expert and 2 shared experts per token. This design maintains high performance while controlling inference costs. Unlike models specialized in a single modality or task, Inkling aims for balanced performance across text, image, and audio tasks, striving to become a universal inference core for enterprise-level agent systems.

Development background: This model was developed by the Thinking Machines Lab team, which has deep expertise in AI safety and capability research and previously launched the FORTRESS security framework. The motivation behind Inkling's development stems from the uneven performance of existing models across diverse tasks. The team aims to reduce the barriers for enterprises to integrate multimodal AI by offering open weights and a配套 fine-tuning platform.

Core value: Inkling addresses the current pain points where models struggle to balance performance across agent, programming, vision, and audio tasks. Through native multimodal design (not post-hoc concatenation) and ultra-long context support (1M tokens), it can perform multiple tasks—such as speech transcription, visual question answering, code generation, and long document analysis—within a single model, reducing system complexity. Additionally, the open weights and Apache 2.0 license allow enterprises to freely customize the model, avoiding vendor lock-in.

Technical features: Key technologies include encoder-free multimodal projection (images are processed through 40×40 pixel blocks using hMLP encoding, while audio is input as discrete tokens via dMel spectrograms), hybrid attention mechanisms (alternating local and global attention layers), and support for multiple quantization precisions such as BF16, MXFP8, and NVFP4. With NVFP4 quantization, the model can run with just 600GB of GPU memory, significantly lowering the deployment threshold. Furthermore, the effort parameter allows developers to dynamically adjust inference depth, flexibly balancing token cost and performance.

2. Key Features

• Multimodal Native Understanding: Processes text, image, and audio inputs simultaneously without requiring external encoders. Images are divided into 40×40 pixel blocks and mapped into a shared latent space through four layers of hMLP; audio is input as discrete tokens in the form of dMel spectrograms. Supports speech transcription, visual question answering, chart reasoning, and long audio analysis. All modalities perform inference within the same decoding space, avoiding information loss caused by modal concatenation.

• Agent and Tool Calling: Built-in multi-framework tool usage capabilities, enabling execution of browser operations, code generation, terminal commands, and multi-turn collaborative development. Achieved a score of 63.8% in the Terminal Bench 2.1 test, making it suitable for building autonomous agent systems and driving complex workflow automation. This capability allows it to act as a backend inference engine, coordinating multiple tools to complete end-to-end tasks.

• Controllable Inference Intensity: Adjusts inference depth via the effort parameter, allowing developers to flexibly balance token cost and performance based on scenario requirements. In low-latency scenarios (e.g., code completion), the effort value can be reduced to minimize the number of generated tokens; in high-accuracy scenarios (e.g., legal analysis), the effort value can be increased to ensure output quality. This design significantly reduces the overall cost of long-process inference, achieving equivalent performance with only one-third the tokens of Nemotron 3 Ultra on the Terminal Bench.

• Ultra-Long Context Handling: Supports up to 1 million tokens of context, thanks to a hybrid attention mechanism (alternating local and global attention layers), maintaining computational efficiency in long-sequence tasks. Suitable for long document analysis, multi-turn conversations, and codebase-level understanding, it can associate reasoning across pages and extract key clauses and risk points.

• Open Weights and Fine-Tuning: Full weights are available on Hugging Face under the Apache 2.0 license, and combined with the Tinker platform, it provides end-to-end fine-tuning and self-training capabilities. Developers can upload custom datasets, define fine-tuning tasks, and even let the model participate in its own iterative optimization, achieving domain adaptation. This openness enables enterprises to customize model behavior while maintaining data privacy.

• Security and Control: Built-in FORTRESS security protection achieved a defense success rate of 78.0% in adversarial testing, with a low false rejection rate. It also supports downstream review tools such as Llama Guard, forming a layered defense system suitable for compliance review needs in enterprise-level deployment. The security mechanism design balances effectiveness with low false positives, reducing interference with normal usage.

3. How to Use

1. Environment Requirements and Prerequisites: Deploying Inkling requires a high-performance GPU cluster. For BF16 precision inference, approximately 2TB of GPU memory is needed (e.g., 8×NVIDIA A100 80GB or 4×H100 80GB). For the NVFP4 quantized version, approximately 600GB of GPU memory is required (e.g., 8×A100 80GB). The recommended operating system is Linux (Ubuntu 22.04+), and you must install Python 3.10+, CUDA 12.1+, and PyTorch 2.1+. It is recommended to use SGLang or vLLM as the inference framework, as they are specifically optimized for MoE architectures.

2. Model Weight Acquisition: Download the weights from the Hugging Face repository (thinkingmachines/Inkling). Use the git lfs or huggingface-cli tool, and execute the command: git lfs clone （link to be updated after official release）

3. Local Deployment for Inference: Taking SGLang as an example, after installation, start the service using the command: python -m sglang.launch_server --model-path /path/to/Inkling --quantization nvfp4 --tp 8. Here, --tp 8 indicates a tensor parallelism degree of 8, suitable for multi-GPU environments. After starting, you can interact with the model via HTTP API, which supports OpenAI-compatible interfaces. If GPU memory is insufficient, you can try MXFP8 quantization or reduce the parallelism, but be aware that this may result in a performance drop.

4. API Integration and Online Experience: Access Tinker Playground (（link to be updated after official release） AI, Fireworks) to integrate and experience the model online. When integrating, use the tml-renderers package to quickly implement tool calling functionality. This package encapsulates interfaces for browser operations, code execution, and more.

5. Fine-tuning and Customization: Upload your training dataset to the Tinker platform (supporting JSONL format), write a fine-tuning task configuration (including learning rate, batch size, number of training epochs, etc.), and submit the training job. The platform provides support for distributed training. After training is complete, you can download the new weights for deployment. Inkling's open weights allow for full-parameter fine-tuning or efficient fine-tuning methods such as LoRA, the latter of which has lower GPU memory requirements and is suitable for resource-constrained scenarios.

6. Inference Effort Adjustment: When making API calls or performing local inference, control the depth of inference using the effort parameter. For example, setting effort=0.5 reduces the number of generated tokens, making it suitable for low-latency scenarios; setting effort=1.0 utilizes the full inference capability. This parameter can be dynamically adjusted with each request, enabling a fine balance between cost and performance. It is recommended to test different effort values during the development phase to find the optimal configuration for specific tasks.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For scenarios requiring native multimodal understanding (especially audio) and high agent capabilities, Inkling is the preferred choice among open-weight models. Its balanced generalist capabilities allow a single model to handle multiple tasks, including text, image, audio, and tool calling, reducing system complexity. If agent programming (Terminal Bench) is a core requirement and visual capabilities are not critical, GLM 5.2 performs better in this aspect, but note that it only supports text and image, and has a fixed inference mode.

For pure text tasks or deployments with limited resources, smaller models like Nemotron 3 Ultra may be more cost-effective, but they lack the flexibility provided by multimodal support and effort parameter adjustment. Inkling's effort parameter gives it a unique advantage in cost-sensitive long-running tasks, allowing developers to dynamically adjust inference depth based on their budget. Overall, Inkling is suitable for enterprise users seeking balanced multimodal capabilities, emphasizing security and controllability, and having a certain level of hardware budget; individual developers or small teams can first experience it through the Tinker Playground online, then assess the feasibility of local deployment.

6. Editor's Summary

Inkling demonstrates several noteworthy innovations in terms of technological advancement. First, its encoder-free multimodal architecture directly projects images and audio into a shared latent space, avoiding information alignment loss commonly found in traditional concatenated multimodal models. This design is relatively rare among open-weight models. Second, the combination of MoE architecture with a hybrid attention mechanism activates only 41 billion parameters out of a total of 975 billion, achieving a balance between scale and efficiency. The "effort" parameter further enables dynamic control over the depth of inference, a feature that has yet to be widely adopted in similar models.

In terms of practical value, Inkling's balanced generalist capabilities directly address the core needs of enterprise-level applications — a single model can handle multiple tasks, reducing system maintenance costs. Its open weights and Apache 2.0 license also remove barriers for commercial use, and when combined with the Tinker platform's fine-tuning capabilities, enterprises can perform domain adaptation while safeguarding data privacy.

In terms of target users, Inkling is primarily aimed at enterprise AI teams, research institutions, and developers who need to build agent systems, with certain hardware resources. For individual developers, the online Playground offers an accessible entry point, although local deployment requires significant GPU memory. Looking ahead, as the community ecosystem matures and more third-party tools become available, Inkling has the potential to establish its own ecosystem in areas such as agents and multimodal understanding. However, it should be noted that its Chinese capabilities have not yet been publicly validated, and adaptation for Chinese scenarios may require additional efforts. Overall, Inkling has taken an important step forward in the openness and balance of multimodal foundation models, offering the industry a compelling option to consider.

7. Application Scenarios

• Enterprise Agent Hub: As a backend inference model, it drives agent systems that enable multi-tool collaboration in customer service, data analysis, and code generation. By fine-tuning to align with internal enterprise workflows, Inkling can uniformly process text queries, image recognition, and voice commands, reducing latency caused by switching between multiple models. Its effort parameter allows for reducing inference depth during peak hours to control costs, while increasing precision for complex tasks.

• Multimodal Content Moderation: Simultaneously analyze text, image, and audio content for efficient compliance checks on social platforms and live streaming services. Inkling's native multimodal capabilities enable it to detect text violations, sensitive image content, and inappropriate speech in audio, combined with FORTRESS security protection, offering a one-stop moderation solution. Support for millions of tokens allows for comprehensive analysis of long live streams or chat histories.

• Interactive Educational Assistant: Based on voice and visual input, it provides real-time problem-solving, chart explanations, and programming guidance for students. Students can ask questions via voice and upload images of problems, and Inkling will understand and generate step-by-step explanations. Its ultra-long context support enables the full loading of entire textbooks, facilitating cross-chapter knowledge connections, making it ideal for personalized learning scenarios.

• Low-latency Programming Assistant: In low-effort mode, it completes code completion, terminal command generation, and lightweight bug fixes with extremely low latency, suitable for embedding in IDE plugins. For simple tasks, reducing the effort value can lower token consumption and improve response speed; for complex refactoring, increasing the effort value ensures code quality. This flexibility makes it applicable to the entire programming assistance workflow, from quick completion to in-depth analysis.

• Long Document Legal/Financial Analysis: Using support for millions of tokens, it performs cross-page reasoning on contracts, research reports, and financial statements to extract key clauses and risk points. Lawyers or analysts can upload entire documents, and Inkling can understand the logical flow of the content, identifying conflicting clauses or abnormal data, significantly improving document review efficiency.

8. FAQ

Q: Is Inkling fully open source? What is the license for the weights?
A: The full weights of Inkling are open-sourced via Hugging Face under the Apache 2.0 license, allowing commercial use, modification, and redistribution. The model code and inference scripts are also open source, but some tools on the Tinker platform may include proprietary components. The Apache 2.0 license provides clear legal protection for enterprise deployment.

Q: What hardware configuration is required to run Inkling?
A: Inference with BF16 precision requires approximately 2TB of VRAM, and it is recommended to use 8×NVIDIA A100 80GB or 4×H100 80GB. The NVFP4 quantized version reduces VRAM requirements to about 600GB, and can be run with 8×A100 80GB or 4×A100 80GB with NVLink. Consumer-grade GPUs (such as RTX 4090 24GB) cannot run it directly, but it can be used via cloud services or third-party inference APIs.

Q: How can Inkling be fine-tuned? Does it support LoRA?
A: End-to-end fine-tuning can be performed via the Tinker platform by uploading a dataset in JSONL format and configuring training parameters. The platform supports both full-parameter fine-tuning and efficient methods such as LoRA. LoRA requires less VRAM and is suitable for resource-constrained scenarios. Fine-tuned weights can be downloaded and used for local deployment.

Q: How is Inkling's Chinese language capability?
A: The official public benchmarks mainly focus on English tasks, and systematic evaluations of Chinese capability have not yet been conducted. The model may include a certain proportion of Chinese text in its pre-training data, but its actual performance must be validated by users themselves. For Chinese use cases, it is recommended to first test via the Playground before deciding to deploy it in production.

Q: How effective is Inkling's 1M token context in practical use?
A: The official claim is that it supports up to 1 million tokens, but the actual usable length is limited by the inference framework and VRAM. Processing input with a million tokens under NVFP4 quantization requires substantial VRAM, and inference speed will significantly decrease. It is recommended to select an appropriate context length based on the task requirements. While the hybrid attention mechanism maintains good computational efficiency for long sequences, the practicality of ultra-long context still needs to be tested in real-world scenarios.

9. Project Links

• Project Website: https://thinkingmachines.ai/news/introducing-inkling/
• HuggingFace Model Repository: https://huggingface.co/thinkingmachines/Inkling
• Technical Paper (Model Card): https://thinkingmachines.ai/model-card/inkling/
• Tinker Playground (Online Experience): https://tinker.thinkingmachines.ai/playground
• Thinking Machines Lab Official Organization: https://thinkingmachines.ai