Today, I was messing around with text embeddings and stumbled upon this thing called “te-ranking”. I’d heard about it before, but never really dug in. So, I thought, why not give it a shot and see what all the fuss is about?
First things first, I needed to get my hands on it. I found it on the internet, and it suggested to try it with “BAAI/bge-large-en-v1.5”, So I went with that one. Good old pip install got that sorted out pretty quick.
Getting Started
Once I had the library installed, I needed some data to play with. I grabbed a few sentences – nothing fancy, just some random stuff I found online. I wanted to see how it would rank these sentences based on a query.
Here’s the stuff I threw at it:
- “The quick brown fox jumps over the lazy dog.”
- “The weather is nice today.”
- “I need to go grocery shopping.”
- “Dogs are better than cats.”
- “This is an example sentence.”
My query was simple: “What animals are mentioned?”
Putting It to the Test
I followed the basic example, creating a list of documents and my super simple query. Then I did model initialized with those files above, It felt like I was about to witness some magic.
I typed some code to encode my query and documents into vector,then use a function to calculate the score between query and documents.
The Results
Boom! The model spit out a ranked list. It looked something like this:
- score:0.67, text: “The quick brown fox jumps over the lazy dog.”
- score:0.45, text:”Dogs are better than cats.”
- score:0.12, text: “This is an example sentence.”
- score:0.08, text:”I need to go grocery shopping.”
- score:0.07, text:”The weather is nice today.”
It actually worked! “The quick brown fox…” sentence was at the top, followed by “Dogs are better than cats.”. The others, which weren’t really relevant, were way down the list. I gotta say, I was pretty impressed.
Overall, my little experiment with te-ranking was a success. It’s a simple way to get started with ranking text based on relevance, and it seems to work pretty well, even with my basic example. I can see how this could be useful for things like search engines or recommendation systems. Maybe I’ll explore it more in the future!
