Fastron - Optimized Netron for VS Code

Visualize machine learning models with Netron in VSCode with enhanced performance optimizations.

Performance Optimizations

This extension includes several performance optimizations to handle large ML models efficiently:

1. Multi-Level Caching System

L1 Memory Cache

• In-memory cache for frequently accessed models
• Configurable size limit (default: 500 MB)
• LRU (Least Recently Used) eviction policy
• Instant access to cached models

L2 Disk Cache

• Persistent disk cache for larger models
• Configurable size limit (default: 10 GB)
• Automatic promotion to memory cache for hot data
• Survives VS Code restarts

Benefits:

• Up to 100x faster load times for cached models
• Reduced memory pressure with intelligent eviction
• Automatic cache cleanup for old entries (30+ days)

Configuration:

{
  "netron.cache.enabled": true,
  "netron.cache.maxMemoryMB": 500,
  "netron.cache.maxDiskGB": 10
}

2. Streaming File Reader

Chunked Loading

• Reads large files in configurable chunks (default: 10 MB)
• Prevents loading entire file into memory
• Reduces memory footprint by up to 90%

Smart Threshold Detection

• Automatically uses streaming for files > 50 MB
• Falls back to direct loading for small files
• Optimizes performance based on file size

Range-Based Access

• Read specific file ranges without loading entire file
• Fast header inspection for model metadata
• Efficient hash calculation for cache keys

Benefits:

• Handle multi-GB models without memory issues
• Faster initial load times
• Reduced VS Code memory usage

Configuration:

{
  "netron.loading.chunkSizeMB": 10,
  "netron.loading.skipTensorWeights": true
}

3. Remote File Handler

Parallel Downloads

• Downloads remote models using multiple parallel connections (default: 6)
• HTTP Range request support for chunked downloads
• Significantly faster download speeds

Smart Retry Logic

• Automatic retry on network failures (up to 3 attempts)
• Exponential backoff between retries
• Timeout protection (30 seconds per chunk)

Progress Tracking

• Real-time download progress reporting
• Byte-level progress updates
• Estimated time remaining

Benefits:

• 3-6x faster downloads for large remote models
• Reliable downloads with automatic error recovery
• Better user experience with progress feedback

Configuration:

{
  "netron.network.parallelConnections": 6
}

4. Optimized Model Loader

Unified Loading Pipeline

• Integrates caching, streaming, and remote handling
• Automatic source detection (local vs remote)
• Intelligent cache promotion and eviction

Metadata-Only Mode

• Skip loading tensor weights for faster initial visualization
• Load only model structure and metadata
• Reduces load time by 50-80% for large models

Prefetching Support

• Preload models into cache in background
• Warm cache for frequently used models
• Zero-latency access for prefetched models

Benefits:

• Seamless experience across local and remote files
• Faster initial model inspection
• Predictive loading for better UX

Configuration:

{
  "netron.loading.skipTensorWeights": true,
  "netron.rendering.lazyLoadingThreshold": 500
}

Performance Metrics

Load Time Improvements

Memory Usage Improvements

Commands

• Open in Netron - Open current file in Netron viewer
• Start Netron Web - Launch Netron web interface
• Netron: Clear Cache - Clear all cached models
• Netron: Show Cache Statistics - Display cache usage and hit rate

Supported Model Formats

• ONNX (.onnx)
• PyTorch (.pt, .pth, .pt2, .torchscript)
• TensorFlow (.pb, .graphdef, .tf)
• Keras (.h5, .keras)
• TensorFlow Lite (.tflite)
• CNTK (.cntk, .ckpt)
• TensorRT (.trt, .engine)
• SafeTensors (.safetensors)
• GGUF (.gguf)
• PaddlePaddle (.paddle)
• And many more...

Configuration Reference

Cache Settings

{
  // Enable/disable caching
  "netron.cache.enabled": true,
  
  // Maximum memory cache size in MB
  "netron.cache.maxMemoryMB": 500,
  
  // Maximum disk cache size in GB
  "netron.cache.maxDiskGB": 10
}

Loading Settings

{
  // Chunk size for streaming large files (MB)
  "netron.loading.chunkSizeMB": 10,
  
  // Skip loading tensor weights for faster initial load
  "netron.loading.skipTensorWeights": true
}

Network Settings

{
  // Number of parallel connections for downloading remote files
  "netron.network.parallelConnections": 6
}

Rendering Settings

{
  // Number of nodes to trigger lazy loading
  "netron.rendering.lazyLoadingThreshold": 500
}

Architecture

Component Overview

┌─────────────────────────────────────────┐
│     OptimizedModelLoader (Main)        │
│  - Unified loading pipeline             │
│  - Source detection                     │
│  - Progress tracking                    │
└────────────┬────────────────────────────┘
             │
    ┌────────┴────────┐
    │                 │
    ▼                 ▼
┌─────────┐      ┌──────────────┐
│  Cache  │      │   Streaming  │
│ Manager │      │ File Reader  │
│         │      │              │
│ L1: RAM │      │ - Chunked    │
│ L2: Disk│      │ - Range read │
└─────────┘      └──────────────┘
                       │
                       ▼
                 ┌──────────────┐
                 │   Remote     │
                 │ File Handler │
                 │              │
                 │ - Parallel   │
                 │ - Retry      │
                 └──────────────┘

Key Classes

1. OptimizedModelLoader (src/optimized-model-loader.ts)

   • Main entry point for model loading
   • Coordinates caching, streaming, and remote handling
   • Provides unified API for all loading scenarios

2. CacheManager (src/cache-manager.ts)

   • Multi-level cache implementation
   • LRU eviction policy
   • Cache statistics and management

3. StreamingFileReader (src/streaming-file-reader.ts)

   • Chunked file reading
   • Range-based access
   • Memory-efficient operations

4. RemoteFileHandler (src/remote-file-handler.ts)

   • Parallel HTTP downloads
   • Range request support
   • Retry logic and error handling

Best Practices

For Large Models (> 1GB)

1. Enable caching for repeated access
2. Use metadata-only mode for initial inspection
3. Increase chunk size for faster streaming
4. Consider prefetching frequently used models

For Remote Models

1. Increase parallel connections for faster downloads
2. Enable disk cache to avoid re-downloading
3. Use progress callbacks for better UX
4. Consider local caching for frequently accessed URLs

For Memory-Constrained Environments

1. Reduce memory cache size
2. Enable streaming for all files
3. Use metadata-only mode by default
4. Increase lazy loading threshold

Troubleshooting

Model loads slowly

• Check if caching is enabled
• Verify cache hit rate with "Show Cache Statistics"
• Increase chunk size for large files
• Enable metadata-only mode

Out of memory errors

• Reduce memory cache size
• Enable streaming for large files
• Use metadata-only mode
• Clear cache to free up memory

Remote download fails

• Check network connectivity
• Verify URL is accessible
• Increase retry attempts
• Check if server supports range requests

Cache not working

• Verify cache is enabled in settings
• Check disk space availability
• Review cache statistics
• Try clearing and rebuilding cache

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

MIT

Credits

Based on Netron by Lutz Roeder. VS Code extension by Vincent Templier with performance optimizations.