Skip to content
Discussion options

You must be logged in to vote

I have found solution to these problems: it is to force mmq pipeline usage (build with -DGGML_CUDA_FORCE_MMQ=1), if it didn't help for the first disable peer copy (will also disable pipeline parallelism, build with -DGGML_CUDA_NO_PEER_COPY=ON)

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by pl752
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
1 participant