Kristian Vybiral

📊 phospho first technical paper is out! We are releasing Intent Embed, an intention embedding model for text messages! 🤖💬 Intent Embed is designed to capture and represent user intention within dense embedding vectors. Unlike traditional text embedding models that focus on semantic or syntactic aspects, Intent Embed zeroes in on the underlying user intent, making it a powerful tool for developers and machine learning engineers working on LLM-based applications. Key highlights from our report: ✅ Demonstrated effectiveness in capturing user intent from complex inputs 📈 Superior performance compared to industry-standard models like OpenAI's text-embedding-3-small 🌐 Generates 1536-dimensional vectors, ensuring compatibility with existing vector infrastructure 🛡 Robust to noise, typing errors and adversarial prompts 🔍 Potential use cases include user request classification, out-of-topic exclusion, and user message analysis 🌟 We believe Intent Embed has the potential to improve how developers and ML engineers approach user-centric applications, contributing to more effective and intuitive LLM interactions. 🤫 We already use it extensively for our user intent clustering in the phospho platform. Of course, you can access it via the phospho API (link in the comments). 📰 Read the full technical report below!

146

17 Comments

Yes! Thank you. I strive for a fixed reroll count in my comparisons, but I also really like what you did - including "how many times did I try before I stopped". This later information contains some complex creator knowledge, like, "how likely am I to get one better than this, given that it is a good enough, artifact free video I'm willing to use" ... Also relevant ... I find my reroll count to be about 5 when I want something good and don't care what it is (think: visual motion, non narrative), but that it edges to 20 when I, as a creator, have a specific vision and I want the ai model to submit to my vision, not visa versa. I think this later is actually shockingly important information. More and more creators will use gen AI. If the tolerance for error is truly Gaussian distributed about the data, then all bias is the data bias. However it gets weirder fast. User preference, or LORA can effectively inject a runtime preference, but aggregate DPO techniques could inject preference as a massive scale (think new model release). Again, maybe ok when we get a visual style. But we reveal our own bias, and might sink further into an AI content bubble, a subspace of latent content (and maybe a model creator has an incentive to couple some other bias to their training data ... ) ultimately creator tolerance / intolerance of that bias intersects with "am I willing to use this one, or do I want to try again?" It seems simple, but the choice of rerolling or not, is going to be crux.

6

7 Comments

Unlike many other MMMs, we publish all of our model results into our platform so that our clients can go out and validate them. We’ll never hide these results. But what happens if the output of our MMM doesn’t match the results from a client’s lift tests? First, an important point – to know whether your model is or isn’t working, you need to be able to validate it. This is hugely important to us, and it’s why we’ve made backtest results available in the Recast app itself so our customers can actually look at them and come to us if something looks off. Because let’s be honest, MMM is an extraordinarily difficult statistical challenge. And while I think Recast does far and away the best across the MMM industry, there are still times that we might miss. If we over/under-predict one time, the model will adjust so that we don’t make the same mistake twice. Now, if backtests are off systematically, that’s an indication that something may be wrong in the configuration of the model – and our customer data science team and our modeling team will go in and take a harder look. This is almost always case by case, but there are a couple of things that we generally look for first: - Is there something the model doesn't know about? I remember a client once had raised prices… but didn’t tell us. Once we added a pricing context variable into the model that problem went away. - Is the model just flat-out wrong? Maybe we aren’t capturing seasonality well, maybe channels are hitting their upper or lower bound… it can be a number of things. This requires that we go into model investigation mode in order to get the model into a good place – this is obviously very important to us so, in the few instances where this happens, it becomes a top priority to get it right!

11

👏 important step forward in understanding "atomic" networks of concepts and features representation inside production grade foundation LLms, the implications are fascinating 1) can we imagine training millions of tiny LLms each representing a feature fine tuned for a specific task 2) can they be then "transplanted" into an existing network? 3) can a larger network be assembled from many parts ? let me know your thoughts

9

Now good folks at Anthropic have released "Golden Gate Bridge LLM" on Claude.ai so we can all play with it, it's spooky how every question is answered with "Golden gate bridge" as the central anchor so if you ask "What's the best coffee in Austin, Texas" is says "There are several iconic spots to get fantastic coffee in San Francisco, including: Golden Gate Bridge - This famous suspension bridge is perfect for photo opportunities and is one of the most visited attractions in the San Francisco area. It spans the Golden Gate strait and connects the city to the Marin headlands." - self-centered to the extreme :) This shows how safety and red-teaming in production grade LLMs is super critical to get right and tweaking few "features" can drastically alter the responses

7

This is more of a "talking to myself in public" thing than any kind of fully fleshed out hypothesis, so take this with the appropriate grain(s) of salt. But if I were interested in AI and were looking for an angle to explore that might yield something interesting, I'd start working on learning "Multiscale Mathematics" (also known as "Multiscale Modeling"). This is rooted in my personal belief that there has to be a "multi-scale" aspect to intelligence because there is such an aspect inherent in our underlying physical reality.

2

Great piece about designing user interfaces with LLMs. Two popular approaches that strike me as misguided: 1. A single prompt that asks the user "what do you want?" and spits out the result on the other end. Obvious examples are image generation tools like DALL-E and Midjourney, but more recently I've seen an increasing number of efforts to apply this to other areas like product configuration or interior design. It's a nice idea, but it simply isn't realistic to expect a user to formulate the sum total of their desires in a sentence or two. Nor is it realistic to expect (with current tech at least) to nail the design even if the user is able to specify it adequately. 2. A chat box slapped onto a traditional UI. While there is definitely a need for both in Conversational UI, there needs to be more interplay between the two than that. Ideally, aspects of the traditional UI make it into the chat window (e.g. rich widgets), and aspects of the chat make it into the traditional UI. Needless to say, we've been experimenting actively with both of these aspects of true Conversational UI in our Conversational Configurator efforts.

10

1 Comment

Gemini 2.0 handles complex math and planning tasks previously out of reach of LLMs. You can turn on "code generation" in AI Studio and in the APIs. For context, let's talk about function calling and voice AI agents. (I spend a lot of time working on voice AI.) Voice AI agents use function calls for: - Interacting with external systems specific to particular use cases and deployment contexts. - A wide range of RAG-like information lookup tasks. - Conversation persistence, including complex actions like saving a privacy-redacted version of a conversation. - Script following. Some of these uses were obvious before we had built and deployed production voice AI systems. But some of them weren't. (Script following, for example.) We had to build experiments, and then production systems, to figure out what features and tooling these new systems really need. I'm convinced that code generation will be as important as function calling, once we've accumulated some experience building with Gemini 2.0 and similarly capable new LLM versions. Credit for this insight goes entirely to Shrestha Basu Mallick, who shepherds work at Google on the Gemini APIs and AI Studio. When Shrestha first told me that really good code generation was going to make a lot of new things possible, I didn't really get it. But after playing with code generation for a few weeks ... I still don't fully get it, but in a good way, now. It's clear that there are some obvious things to explore: - Context-aware math on demand driven by natural language. - Generative UI. - Better "binding" behavior with function calling. But I'm also convinced now that there is a lot of unexplored, non-obvious (at least to me, yet) territory. Let's take that last one — improved function calling performance. Function calling and code generation go together like peanut butter and jelly. Functions often take arguments that are fairly complicated. With code generation enabled, Gemini has access to this new pattern: natural language ➡️ code that creates argument structs for functions ➡️ multiple function calls Think about how powerful this is. The ability to use even very basic code structures — loops, for example — massively expands what LLMs can do. In fact, my impression is that function calling in Gemini 2.0 is now trained as a subset of the model's code generation capabilities. This feels quite different than how function calling works in other models that I have access to. In other models, function calling feels like a close cousin to structured output. Which is great. But this new "modality" is even better. (More on the prompt in this post, and the meta-prompt for this post, here: https://lnkd.in/gycKtnNX)

49

9 Comments

How can AI Agents help drive R&D productivity? This presentation from Aviv Regev shows how Roche uses agents to accelerate R&D outcomes. While Aviv takes the perspective of a computational biologist and my focus is operational excellence, our objective is the same: Use AI to improve biopharma R&D productivity. AI Agents like Roche’s Braid work to: 1. reveal more targets and increase identified leads in the pipeline 2. enrich qualification and optimization of molecules and modalities Unipr AI Agents can work in complement to: 1. compress phase dwell times to accelerate pipeline progression 2. improve portfolio prioritization to terminate risky programs early 3. lower costs through better resource, time, and budget allocation With AI, the future of biopharma innovation is more promising than ever!

7

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

There's a new AI image model in town It's pretty good. Flux is a new open-source AI model from some of the ex-stability AI team. Is it as good as current (IMO) leader Midjourney? It's high-quality, free - and at first tests does text generation than the others in the market - including Midjourney, Dalle, Firefly, Ideogram. Also those pesky fingers are better as well. Here's the one-shot prompt: 'Person holding up a cardboard sign saying G3NR8 is testing an open-source AI model's capability. Is it good, bad or ugly?' You can see the result in the image. You can have a play with it here: https://lnkd.in/eYuPV6hg Let me know what you think. #ai #image #models

15

3 Comments

Thought provoking quote about a new wave of consumer AI apps - "conforming software" - by Scott Belsky. "a pattern we'll see with new wave of consumer AI apps:" "the more you use the product, the more tailor made the product becomes for you. beyond memory of past activity and stored preferences, the actual UI and defaults and functionality of the product will become more of what you want and less of what you don't." "it's a new type of "conforming software" that becomes what you want it to be as you use it"

11

5 Comments

Unpopular opinion: AI won't revolutionize design. It will make good designers great. And great designers unstoppable. Here's why: • AI accelerates ideation • Humans guide the process • AI optimizes for engagement • Humans ensure mission alignment • AI analyzes user data • Humans translate insights into impact At our social impact design studio, we use AI tools to complement human-centered design strategy. For us, it's not about the tech. It's about the mission and the impact. But don't get me wrong, AI is a powerful tool that allows you to produce more work in less time. However, the real magic in design happens when we create with empathy, compassion, creativity, and a deep understanding of who and what we're designing for and how to drive real change. That's still 100% human. What's your take on AI's role in design? I'd love to hear your thoughts. --- Follow me for more insights and stories on how to leverage design for positive change. #AI #SocialImpact #UXDesign #WebDesign #NonprofitTech #AIForGood

1

3 Comments

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

Sure, the opportunity to build AI Agents is there, but existing SaaS companies have a major advantage: their distribution channel. It’s easier to experiment providing agentic solutions (under their tailored business model) if you have an existing customer base and deep domain expertise, otherwise brace for 360° disruptive competition.

10

1 Comment

5 hours long Lex and Dario Amodei, the co-founder and CEO of Anthropic, a company behind Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus LLMs, speak about how AI is shaping the future and how it could impact the way we live and work. They discuss how far AI can go. Amodei thinks AI can match human intelligence and even surpass it in areas like biology, where complexity overwhelms humans. But in other areas, like speech recognition or conflict resolution, AI might hit limits close to ours. One of the challenges is the fact of running out of quality data and the cost of scaling compute power. Enjoy the entire talk! https://lnkd.in/dRhpafzU

9

2 Comments

🌟 Explore Claude 3.5 Haiku: Anthropic's Latest AI Model for Everyone Anthropic has launched the Claude 3.5 Haiku model, accessible for all users via the Claude chatbot. This model outperforms larger models on key benchmarks while maintaining a budget-friendly pricing structure. 📈 Performance Highlights of Claude 3.5 Haiku With low latency and a 200,000-token context window, it excels in handling extensive datasets and evaluating financial documents. 🔗 Accessing Claude 3.5 Haiku Claude 3.5 Haiku integrates seamlessly with Claude Artifacts for enhanced interactive capabilities, albeit lacking web browsing features. 💡 Subscription Options Available for free or via a $20/month Pro plan, it promises impressive performance at a reasonable cost. For more insights, dive into this groundbreaking AI model! #AI #Anthropic #Claude3 #ArtificialIntelligence #Innovation Read more: https://lnkd.in/gj25nuxN

2

Looks like there's a new leader in inference speed: SambaNova. I just tried it and got over 1,000 tokens per second using Llama-3 8B. Looks faster than Groq. You can try it here: https://fast.snova.ai/. Benchmark is here: https://lnkd.in/g95Zv7-C

2

I had the reverse reaction. I don't think anyone will actually search that way. What I think is going to happen is we'll see more aggressive (and more interesting) query refinements and facets. People will still start with something like "yoga studios near me" or maybe "yoga studios near me with intro offers" and then click a couple of facets to refine their query. It's going to be more of an ongoing conversation, not a one-time shot. "yoga studios near me" "4-stars and above only" "OK, which ones are within a 10 minute walk" "show me if any of those have intro offers" Most of the features Google showed today had much higher cognitive load to figure out how to ASK the AI than just executing the task yourself, or breaking the task into smaller chunks.

28

7 Comments