Skip to content

Is it possible to use a LoRA with a K-quantized model on GPU #936

@LostRuins

Description

@LostRuins

I would've sworn it was possible at one point, but perhaps I misremembered it. Currently it doesn't work, as cuda get_rows and vulkan ggml_copy do not support K quants.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions