where to find downloaded models ?

[zoldaten]

inference put models to temp folder ?
in user/cache cant find them.

[xenova]

Hi there 👋 Generally, the models are added to the user's browser cache using the Web Cache API. You can find this in dev tools as follows:
Application -> Cache storage -> transformers-cache

[Nithur-M]

Hi @xenova, any way to store this to indexed db so that I don't need to download the models for each session? or is there any other workaround? Thank you.

[hpssjellis]

Hi @xenova, any way to store this to indexed db so that I don't need to download the models for each session? or is there any other workaround? Thank you.

Typically with Web-LLM from the cache the models should auto load. I will do some testing.

[hannes-sistemica]

Hello, I was also trying to find a way to not only cache the models, but also make multiple available at once and switch between them, without always downloading them. You can see the cached model here (in browser console):

async function listCachedModels() {
    // Get all cache names
    const cacheNames = await caches.keys();
    
    // Look for transformers cache
    const transformersCache = await caches.open('transformers-cache');
    
    // Get all cached requests
    const requests = await transformersCache.keys();
    
    // Group by model name
    const modelFiles = {};
    for (const request of requests) {
        const path = request.url;
        const modelName = path.split('/').slice(-2)[0];  // Get the model name from path
        
        if (!modelFiles[modelName]) {
            modelFiles[modelName] = [];
        }
        modelFiles[modelName].push(path);
    }
    
    return {
        models: Object.keys(modelFiles),
        modelDetails: modelFiles
    };
}

// Use it with async/await in console
const result = await listCachedModels();
console.log('Cached Models:', result.models);
console.log('Model Details:', result.modelDetails);

It would be great to store the models in indexDB. I am already thinking to build a model loader from/to indexDB/cache, but that would always have the active model within cache and indexDB in parallel.

[hannes-sistemica]

Actually I found out that multiple models can be downloaded and cached at the same time. So maybe no need for indexDB?