sentence-transformers/all-MiniLM-L6-v2
This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.
Benchmarks
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Quality
ndcg at 10 0.4698
map at 10 0.4073
mrr at 10 0.4632
Performance A10G b1 c16
Corpus TPS 1.7K
Corpus p50 1.1s
Query TPS 512
Query p50 327.0ms
Performance L4 b1 c16
Corpus TPS 37.0K
Corpus p50 53.3ms
Query TPS 2.8K
Query p50 49.8ms
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
Quality
ndcg at 10 0.3288
map at 10 0.2577
mrr at 10 0.2885
Performance A10G b1 c16
Corpus TPS 1.1K
Corpus p50 750.0ms
Query TPS 299
Query p50 296.9ms
Performance L4 b1 c16
Corpus TPS 17.1K
Corpus p50 50.1ms
Query TPS 1.8K
Query p50 45.5ms
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Quality
ndcg at 10 0.3687
map at 10 0.2914
mrr at 10 0.4451
Performance A10G b1 c16
Corpus TPS 1.9K
Corpus p50 1.3s
Query TPS 587
Query p50 345.4ms
Performance L4 b1 c16
Corpus TPS 46.3K
Corpus p50 52.4ms
Query TPS 3.7K
Query p50 43.8ms
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
Quality
ndcg at 10 0.6560
map at 10 0.5883
mrr at 10 0.5874
Performance L4 b1 c16
Corpus TPS 128.8K
Corpus p50 58.9ms
Query TPS 4.3K
Query p50 59.1ms
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.3160
map at 10 0.1105
mrr at 10 0.5040
Performance L4 b1 c16
Corpus TPS 81.3K
Corpus p50 54.7ms
Query TPS 1.5K
Query p50 44.7ms
NanoFiQA2018Retrieval
Smaller subset of the FiQA financial QA dataset
Quality
ndcg at 10 0.4774
map at 10 0.3931
mrr at 10 0.5476
Performance L4 b1 c16
Corpus TPS 44.2K
Corpus p50 56.1ms
Query TPS 2.8K
Query p50 49.9ms
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Quality
ndcg at 10 0.2164
map at 10 0.1294
mrr at 10 0.3594
Performance L4 b1 c16
Corpus TPS 55.3K
Corpus p50 51.8ms
Query TPS 4.1K
Query p50 43.1ms
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Quality
ndcg at 10 0.6451
map at 10 0.5959
mrr at 10 0.6047
Performance L4 b1 c16
Corpus TPS 75.7K
Corpus p50 52.8ms
Query TPS 5.8K
Query p50 44.5ms
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Quality
ndcg at 10 0.8396
map at 10 0.8117
mrr at 10 0.8117
Performance L4 b1 c16
Corpus TPS 58.9K
Corpus p50 56.0ms
Query TPS 65.7K
Query p50 61.5ms