Vectorizer for Ollama: Add Ollama support for vectorizer

[theodufort]

What problem does the new feature solve?

Add Ollama support for vectorizer

What does the feature do?

Allows to use self hosted Ollama instance to process embeddings.

Implementation challenges

No response

Are you going to work on this feature?

None

[Askir]

We are on it already 😄
#227

[orenmizr]

wait, so if i want to use local embedding model - that means that i can't right now ? was trying to make it work with nomic-embed-text but simply could not make it work.

[theodufort]

wait, so if i want to use local embedding model - that means that i can't right now ? was trying to make it work with nomic-embed-text but simply could not make it work.

You can actually, I am using it via the ollama.embed function which works perfect. The only caveat is I have to batch generate the embeddings like so:

CREATE OR REPLACE FUNCTION public.generate_embeddings_title_batch()
 RETURNS void
 LANGUAGE plpgsql
AS $function$
DECLARE
    processed_count INTEGER;
    start_time TIMESTAMP;
    end_time TIMESTAMP;
    func_start_time TIMESTAMP;
    func_end_time TIMESTAMP;
    func_elapsed_time INTERVAL;
    elapsed_time INTERVAL;
    avg_speed NUMERIC;
BEGIN
    LOOP
        -- Record the start time for the batch
        start_time := clock_timestamp();

        -- Measure time for the vector generation
        func_start_time := clock_timestamp();

        WITH batch AS (
            SELECT id, title
            FROM books_parted_eng
            WHERE title IS NOT NULL AND title_embedding IS NULL
            LIMIT 1000
        )
        UPDATE books_parted_eng
        SET title_embedding = (SELECT generate_vector(batch.title))
        FROM batch
        WHERE books_parted_eng.id = batch.id;

        -- Record the end time for the vector generation
        func_end_time := clock_timestamp();
        func_elapsed_time := func_end_time - func_start_time;

        -- Record the end time for the batch
        end_time := clock_timestamp();

        -- Count the rows processed
        GET DIAGNOSTICS processed_count = ROW_COUNT;

        -- Calculate elapsed time for the whole batch
        elapsed_time := end_time - start_time;

        -- Calculate average vector generation speed
        avg_speed := CASE
                        WHEN EXTRACT(EPOCH FROM func_elapsed_time) > 0 THEN
                            processed_count / EXTRACT(EPOCH FROM func_elapsed_time)
                        ELSE
                            0
                     END;

        -- Log the stats in a human-friendly format
        RAISE NOTICE 'Rows updated: %, Avg vector generation speed: % rows/sec, Total batch time: % ms, Vector generation time: % ms',
                     processed_count,
                     ROUND(avg_speed, 2),
                     ROUND(EXTRACT(EPOCH FROM elapsed_time) * 1000), -- Convert seconds to milliseconds
                     ROUND(EXTRACT(EPOCH FROM func_elapsed_time) * 1000); -- Convert seconds to milliseconds

        -- Exit the loop if no more rows to process
        EXIT WHEN processed_count = 0;
    END LOOP;

    -- Log completion
    RAISE NOTICE 'All updates are complete.';
END;
$function$
;