Designing Machine Learning ApplicationsDesigning Machine Learning Applications
Home
Discus
Home
Discus
  • Contents
  • Preface

    • About the Author
    • About DMLA
  • Mathematical Foundations

    • Linear Algebra

      • Vector Basics
      • Matrix Basics
    • Calculus

      • Limits, Derivatives, and Differentials
      • Multivariate Functions and Composite Function Differentiation
    • Statistics and Probability

      • Probability Basics
      • Statistical Inference
  • Classical Statistical Learning

    • Linear Models

      • Linear Regression
      • Logistic Regression
      • Regularization and Generalized Linear Models
    • Bayesian Methods

      • Naive Bayes
      • Bayesian Network
      • EM Algorithm
    • Support Vector Machines

      • Support Vector Machine
      • Kernel Trick
    • Decision Trees and Ensembles

      • Decision Trees
      • Random Forest
      • Boosting
    • Unsupervised Learning

      • Clustering
      • Dimensionality Reduction
  • Neural Networks and Deep Learning

    • Neural Network Architectures

      • Fundamentals of Neural Networks
      • Linear Perceptron
      • Multi-Layer Perceptron
      • Forward Propagation
      • Backpropagation
      • Activation Functions and Loss Functions
    • Optimization

      • Gradient Descent
      • Adaptive Optimizers
    • Deep Network Stability

      • Weight Initialization
      • Dropout Regularization
      • Batch Normalization
    • Convolutional Neural Networks

      • CNN Basics
      • AlexNet and the CNN Revival
      • VGG and GoogLeNet
      • ResNet Residual Network
      • Lab: AlexNet Image Classification
    • Generative Models

      • Variational Autoencoder
      • Generative Adversarial Network
      • Lab: DCGAN Image Generation
    • Sequence Models

      • Word Embedding and Representation Learning
      • RNN Fundamentals
      • LSTM and GRU Gating Mechanisms
      • Seq2Seq Sequence Mapping
      • Lab: LSTM Poetry Generation
  • The Language Model Singularity

    • Transformer Architecture

      • Transformer Fundamentals
      • Transformer Evolution and Variants
      • Language Models and Tokenization
      • Lab: Transformer Model Training
    • Pretraining and Fine-Tuning

      • Pretraining Data Engineering
      • Scaling Laws
      • Distributed Training Infrastructure
      • Supervised Fine-Tuning
      • Lab: SFT Model Conversation
    • Alignment Training

      • Reinforcement Learning from Human Feedback
      • Evolution of Alignment Methods
      • Lab: DPO Alignment Training
    • Reasoning Capabilities

      • Chain of Thought and Reasoning Models
      • Test-Time Compute Scaling
      • Inference Efficiency Optimization
      • Lab: LLM Inference Optimization
    • Multimodal Fusion and Safety

      • Multimodal Large Language Models
      • Model Evaluation and Safety
      • Lab: VLM Training
  • AI Infrastructure and Engineering

    • Model Serving

      • Inference Service Architecture
      • Request Scheduling and Batching
      • GPU Resource Management
      • Lab: Deploying LLM Inference Service
    • MLOps Practices

      • Data Versioning
      • Experiment Tracking and Model Registry
      • Hyperparameter Optimization
      • Model Performance Monitoring
      • Drift Detection
  • Agentic Application Systems

    • Vector Retrieval and RAG

      • Embedding and Vector Retrieval
      • Retrieval Quality Evaluation and Optimization
      • Retrieval-Augmented Generation
      • Lab: Building a Knowledge Base Q&A System
    • Building Agent Applications

      • From LLM to Agent
      • Tool Use
      • Planning and Reasoning
      • Memory Systems
      • Agent Collaboration and Communication
      • Orchestration and Fault Tolerance
      • Lab: Research Agent Collaboration System
  • Appendix

    • Building the Sandbox Environment
    • NumPy Practice

      • Data Processing Practice
      • Calculus Computation Practice
      • Probability and Statistics Practice
Designing Machine Learning Applications

Designing Machine Learning Applications

visitorslast commitCInpmlicense: CC BY-NC-SA 4.0author: icyfenix

Start ReadingTable of Contents

About This Document

This document explores how to apply machine learning to production systems, offering software developers a roadmap for organizing their knowledge in the AI era. In spirit, it is similar to my other work, The Fenix Projects. However, this document has a steeper learning curve than Phoenix Architecture. It is recommended to read About Designing Machine Learning Applications first to understand the document's purpose and target audience.

I wrote this document with the following goals in mind:

  • I am a software engineer and a PhD in machine learning. At a time when the software industry is shifting from "hard-coded logic" to "data-driven intelligence," I hope to use this document as a guide to systematically organize my knowledge of machine learning, fill in the gaps, and integrate it all into my own knowledge framework.
  • I believe there is a shortcut to technical growth: not only reading, studying, thinking, and applying, but also articulating and writing. Expressing what you believe you have mastered -- stating it clearly and coherently, explaining it with confidence, making others understand and resolving their doubts, and subjecting your views to the scrutiny and even challenge of others. In this process, you uncover much of the "unknown" lurking behind the "known." There is no true knowledge without action; knowledge without action is not yet true knowledge.

Quick Start

  • Read Online: The online reading address for this document is https://ai.icyfenix.cn/en. The website is hosted on GitHub Pages for storage, uses GitHub Actions for continuous integration to compile and sync Markdown documents from the Git repository to the site in real time, pushes to CDN for accelerated access in China, and uses GitHub Discussions with Giscus for discussion services.

  • Code Sandbox: This document contains numerous interactive code examples that run directly in your browser for demonstrating machine learning algorithms and model training. For Part 1 (Mathematical Foundations) and Part 2 (Classic Statistical Learning Algorithms), a pure CPU environment is sufficient. For deep learning and beyond, some model training experiments require GPU support (currently based on PyTorch with CUDA 13.0). For more details, see Building the Sandbox Environment.

    # Install and run the sandbox environment
    npx @icyfenix-dmla/install@latest
    
  • Run Offline:

    • Deploy an offline site: The document is built with VuePress. If you want to host the documentation site within your organization, use the following commands:
    # Clone the repository
    git clone https://github.com/fenixsoft/dmla.git && cd dmla
    
    # Install project dependencies
    npm install
    
    # Run the website; default address is http://localhost:8080, sandbox address defaults to http://localhost:3001
    npm run local
    
    • For local runtime, running the code examples in the document still requires a Docker image. You can pull the image directly using the one-click sandbox installation command, or build it locally with the following command:
    npm run build:sandbox:all
    
  • Secondary Creation, Distribution, and Publishing: All content in this document cites sources where applicable, with rights belonging to the original authors. All other content, including but not limited to text, images, tables, etc., is original work by the author. This original content is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0). As long as you follow the license terms of attribution, non-commercial use, and share-alike, you may use, modify, adapt, and distribute any part of this document anywhere, in any form, to anyone. See the "License" section of this document for details.

License

  • This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0). Under this license, you are free to share -- copy and redistribute the material in any medium or format -- and adapt -- remix, transform, and build upon the material, provided you meet the following conditions:
    • Attribution: You must give appropriate credit to the original author and source.
    • Non-Commercial: You may not use the material for commercial purposes. For commercial use, please contact the author.
    • ShareAlike: If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

Filing

Website ICP filing: Guangdong ICP 18088957

Last Updated:
Contributors: icyfenix, Claude