[et_pb_section fb_built=”1″ _builder_version=”4.23.1″ _module_preset=”default” global_colors_info=”{}”][et_pb_row _builder_version=”4.23.1″ _module_preset=”default” custom_margin=”-1px|auto||auto||” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.23.1″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.23.4″ _module_preset=”default” custom_padding=”0px|||||” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n
🆕 The new OpenAI embedding models are cheaper and better – what’s not to love? Well, let’s dig a bit deeper. 🤔<\/p>\n
🔍 What exactly are embedding models?\u00a0For each paragraph in your documents, these models create a numerical representation of its meaning. We then use them for efficient searching within documents or for grouping similar documents together (a process known as clustering).<\/p>\n
🌟\u00a0The new models excel at capturing the essence of text:<\/p>\n
🧠 So, using the new models seems like a no-brainer, right?<\/p>\n
⚠️\u00a0Here is the catch:\u00a0when switching to a different model, you must recreate the embeddings for all the paragraphs of all your documents. This might be low-cost, but<\/p>\n
👉 This also points to a more long term challenge of evolving our AI systems. What happens when OpenAI stops supporting the embedding you have chosen? Which begs the questions: wouldn\u2019t it be better to take control of the embeddings by running our own model? Unfortunately it is not clear that there is an open source of the same quality as OpenAI. yet\u2026<\/p>\n
[\/et_pb_text][et_pb_image src=”http:\/\/denayer.com\/blog\/wp-content\/uploads\/2024\/01\/embeddings.png” alt=”The End is Never Near” title_text=”embeddings” _builder_version=”4.23.4″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″][\/et_pb_image][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"
The new OpenAI embedding models are cheaper and better – what’s not to love? Well, let’s dig a bit deeper. <\/p>\n","protected":false},"author":1,"featured_media":3112,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[34],"tags":[],"class_list":["post-3108","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t