I have been trying to recreate a SBERT model based off the original paper. I found a great tutorial to do it and basically followed it, showing you how to train an SBERT model from using a pretrained BERT model. This tutorial however uses MNR Loss, unlike the original paper that uses Softmax loss, but other than that it should be the same process:

I am trying to find similar words to a list of words I have embedded with slight modifications. The goal in the end is to try and build a search tool that can generate a list of single words from an embedding. Reading through the literature though suggests that sentence transformers are not capable of performing this task as they are descriptive in nature and not generative.

I need to create embeddings and do cosine similarity calculations using sentence-transformers/all-MiniLM-L6-v2 in an ‘offline’ environment (huggingface is blacklisted in this environment since it is a community hub). I am having trouble finding info on how to run the model in offline mode. Any ideas?

I’m (more or less) following the Training_quora_duplicate_questions.py example using my own data.