I am excited to see this library. I am trying to use it on Win 11 PC. Initially, I had trouble getting the library to find the location of the executables. Looking at your code, I could understand what you were doing. I would suggest that instead of expecting an "external" folder to connect to llama.cpp, add a "vendor" folder. This change brings your library inline with llama_cpp_python, which also connects to llama.cpp. This will enable the user to better understand how to set up VulkanILM. The missing "external" folder usage requires a user to dig through your code to figure out what to do. Since many users don't have the experience that you and I have, they will decide to not use your library because there is very little information to guide them in how to set it up.
I am excited to see this library. I am trying to use it on Win 11 PC. Initially, I had trouble getting the library to find the location of the executables. Looking at your code, I could understand what you were doing. I would suggest that instead of expecting an "external" folder to connect to llama.cpp, add a "vendor" folder. This change brings your library inline with llama_cpp_python, which also connects to llama.cpp. This will enable the user to better understand how to set up VulkanILM. The missing "external" folder usage requires a user to dig through your code to figure out what to do. Since many users don't have the experience that you and I have, they will decide to not use your library because there is very little information to guide them in how to set it up.