Replies: 11 comments 8 replies
-
|
Now it can be used from HugginFace with the model ID "meta-llama/Meta-Llama-3-8B-Instruct". |
Beta Was this translation helpful? Give feedback.
-
We should test wether this performs better than MPNet (and also the time wise trade-off) and replace the MPNet default if it produces better results |
Beta Was this translation helpful? Give feedback.
-
|
Access to meta-llama/Meta-Llama-3-8B-Instruct model is restricted. Also, it is no longer labeled as "sentence similarity" but "text generation". McGill has an alternative model McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised, but we need to write a new adapter as this model does not utilize SentenceTransformer module. |
Beta Was this translation helpful? Give feedback.
-
|
We need in general a well performing model that can map from other languages (e.g. german) to the english terminologies we are using. Doesn't need to be llama specifically but we need some sort of alternative to MPNet since it can only handle english to english. |
Beta Was this translation helpful? Give feedback.
-
|
I am looking into it. For now, we can test FremyCompany/BioLORD-2023-M as it can handle semantic similarity in a multi-lingual context. This also does not require a new adapter. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Thanks for the research, great work. I think based in this we should probably switch to the text-embedding-3-large model since it appears to perform best in both metrics |
Beta Was this translation helpful? Give feedback.
-
|
Just saw this is openAI based, probably best trade off then is sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2, what do you think? |
Beta Was this translation helpful? Give feedback.
-
|
This makes sense to me as well, it is a great trade-off between overall accuracy and multilingual capability compared to sentence-transformers/all-mpnet-base-v2. |
Beta Was this translation helpful? Give feedback.
-
|
In the last CePPH project meeting someone proposed this one for german/english mapping: Have you seen this one yet? It is also available as part of sentence-transformers library so we could also use this |
Beta Was this translation helpful? Give feedback.
-
|
Cool! I didn't see this, I will try it and extend the table we have. |
Beta Was this translation helpful? Give feedback.
-
|
This requires several modifications to our class and the input.
|
Beta Was this translation helpful? Give feedback.
-
|
@mehmetcanay seen this: Looks promising, we could use this as a wrapper to solve #21 |
Beta Was this translation helpful? Give feedback.
-
|
I didn't see this before, looks very cool and will definitely allow us to simplify. But majority of the models we have tested are not listed in the README of the project, are we sure that it will support the models we want? |
Beta Was this translation helpful? Give feedback.
-
|
Langchains API also looks like a good option: |
Beta Was this translation helpful? Give feedback.
-
|
This looks like a better option as it already supports Huggin Face models and Open AI. Plus the Llama3 model is also implemented and can be used free of charge. |
Beta Was this translation helpful? Give feedback.
-
|
Will we be able to generate embeddings in batches with this API in-between though, or is it going to be a trade-off between simplicity and time required for calculation? |
Beta Was this translation helpful? Give feedback.
-
|
I just checked the article Martin sent us. Apparently MistralAI shares their weights publicly which makes them semi-open source. This could be interesting to the user. |
Beta Was this translation helpful? Give feedback.
-
|
Also OLMo 7B Instruct model regarded as the most open-source model in the same article. The model exists in Hugging Face, but I am not sure whether it can be utilized for Sentence Similarity without doing anything additional. Edit: I am adding the figure as well so we can also see the criteria.
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Beta Was this translation helpful? Give feedback.
All reactions