[#31] Revolutionizing In-App Promotions and Messaging: The Potential of LLMs

Potentially rethinking how in-app messaging could be infused with LLMs

Introduction

In the fast-paced world of digital marketing, delivering personalized, timely, and engaging promotions can significantly impact a company's success. A recent insight from a CXO executive at a leading Southeast Asian super-app shed light on how Large Language Models (LLMs) are transforming their approach to in-app messaging for promotions. This blog post explores the technical aspects and strategic implications of integrating LLMs into promotional systems, considering both the challenges and the potential.

The Status Quo vs. The LLM Revolution

Creating effective in-app promotional messages has traditionally been time-consuming. At the super-app company, it took an average of 100 hours from ideation to deployment, with 95 hours spent on idle time and only 5 hours on actual interaction. This process was inefficient and limited in its ability to deliver truly personalized content at scale.

LLMs offer a promising solution. The executive reported that by leveraging these advanced AI models, they reduced the entire process to just 1.5 hours while improving click-through rates (CTRs) through enhanced personalization. This dramatic improvement raises important questions: How might such a system work? What are the key technical components? And perhaps most importantly, where does the true value of LLMs lie in this context?

Please note:
As an observer of emerging technologies, I find the potential of LLMs in promotional systems compelling. While I don't have firsthand experience implementing such a solution, exploring how an LLM-powered platform might function could yield valuable insights.

This is a conceptual exercise to understand the possibilities of integrating LLMs into marketing. It doesn't claim to represent the complexity of existing implementations but imagines how these technologies could create more personalized and engaging promotional experiences.

I invite all the feedback on the topics explored here to develop these ideas further.

Key Components of an LLM-Powered Promotion System

An effective LLM-powered promotion system would likely include the following components:

1. Data Integration and Preprocessing: The foundation of any effective personalization system is comprehensive and well-organized data. This includes historical transaction data, user preferences, demographic information, app usage statistics, and real-time contextual data.

2. Real-time Feature Engineering: Continuously updating and calculating relevant features is crucial. This includes RFM (Recency, Frequency, Monetary) scores, user segment classifications, and propensity scores for different types of offers.

3. Vector Database and Embeddings: User profiles and promotional content are converted into dense vector representations (embeddings) and stored in a vector database. This allows for efficient retrieval of relevant information.

4. Retrieval-Augmented Generation (RAG) System: When generating a promotion, the system retrieves the most relevant information from the vector database to augment the prompt sent to the LLM.

5. LLM Integration: A fine-tuned LLM receives the augmented prompt and generates personalized message content based on the input.

6. Content Optimization: Historical performance data is used to fine-tune the LLM over time, potentially incorporating reinforcement learning techniques to optimize for metrics like CTR and conversion rates.

7. Approval and Deployment: An automated review system checks for compliance and brand guidelines, with the option for human review of edge cases before deployment to users.

Detailed Workflow: From Trigger to Delivery

Here's a step-by-step breakdown of how an LLM-powered system might operate in practice:

1. Campaign Initiation: Marketing team defines campaign objectives and target audience.

2. User Selection: System identifies users matching campaign criteria.

3. Contextual Data Retrieval:

   • RAG system queries the vector database for relevant user data.

   • Retrieved data includes user preferences, recent behaviors, and current context.

4. Prompt Construction:

   1. System assembles a prompt for the LLM, incorporating:

      • Campaign objectives

      • User-specific data

      • Relevant historical campaign performance

Let's look at some examples of how these prompts might be constructed:

• For a general promotion:

Generate a promotional message for a ride-hailing service with the following parameters:
- User: Male, 28, typically uses the service for airport trips
- Current context: Thursday, 4 PM, user's location is 5 miles from the airport
- Campaign goal: Increase airport ride bookings by 15%
- Tone: Professional but friendly
- Include: Mention of reliability for catching flights

• For a personalized food delivery promotion:

Create an in-app message promoting food delivery with these specifics:
- User: Female, 35, usually orders vegetarian, prefers Italian cuisine
- Recent activity: Browsed Thai restaurants in the last 24 hours
- Current context: Monday, 11:30 AM, user is at their work location
- Campaign goal: Boost lunchtime orders by 20%
- Tone: Casual and enticing
- Include: Mention of a time-limited discount

5. Message Generation:

   • LLM generates multiple message variants based on the prompt.

   • Each variant is tailored to the user's preferences and current context.

   For the ride-hailing promotion, the LLM might generate:

   1. "Airport run? We've got you covered. Book now and enjoy a stress-free ride to catch your flight. Use code FLY15 for 15% off!"

   2. "Time to head to the airport? Our reliable drivers ensure you never miss a flight. Book now and save 15% with code AIRPORT15."

   For the food delivery promotion:

   1. "Craving a taste of Italy for lunch? Get 20% off your favorite veggie pasta, delivered right to your office. Order in the next 30 minutes!"

   2. "Switch up your lunch routine! Enjoy 20% off our new Thai vegetarian options. Perfect for a quick and delicious workday meal."

6. Quality Check:

   • Automated systems check messages for brand consistency and compliance.

   • Optional human review for complex campaigns or randomly sampled messages.

7. A/B Testing Setup (for larger campaigns):

   • System allocates message variants to user segments for testing.

   • For example, in the food delivery campaign:

     • Variant 1 (Italian focus) sent to 50% of users

     • Variant 2 (Thai option) sent to the other 50%

   • System tracks performance metrics for each variant

8. Message Delivery:

   • Chosen message is sent to the user through the app at the optimal time.

   • In our food delivery example, messages would be sent at 11:45 AM to catch the lunchtime crowd

9. Performance Tracking:

   • System monitors key metrics like open rates, click-through rates, and conversions.

   • Data is fed back into the system for continuous improvement.

   • For instance, if the Thai option (Variant 2) shows a 25% higher click-through rate, this information is used to:

     • Immediately adjust the campaign to favor the better-performing variant

     • Inform future prompt constructions for similar campaigns

10. Real-time Adjustment:

    • Based on user interactions, the system can make real-time adjustments.

    • For example, if a user doesn't engage with the food delivery promotion but opens the app again at 2 PM:

      • The system might generate a new prompt:

Create a follow-up message for a user who didn't engage with the lunchtime promotion:
- User: [Previous details]
- Current context: Monday, 2 PM, user opened app but didn't order
- New goal: Promote afternoon snack or early dinner options
- Tone: Friendly and understanding
- Include: Mention of extended discount validity

• The LLM might then generate: "Missed lunch? No worries! Our 20% discount is still valid. Treat yourself to an afternoon pick-me-up or get ahead on dinner. Order now!"

Real-time Updates: Keeping User Profiles Fresh

A key advantage of this system would be its ability to maintain up-to-date user profiles through continuous updates:

1. Event-Driven Updates:

   • User actions (e.g., app opens, searches, purchases) trigger immediate profile updates.

   • These events are streamed in real time to update user embeddings.

2. Incremental Learning:

   • As new events occur, the system slightly adjusts the user's embedding vector.

   • This allows for gradual shifts in user preferences and behaviors to be captured.

3. Periodic Batch Updates:

   • Scheduled (e.g., daily) updates incorporate more complex features or analyses.

   • This might include updated segmentation or lifetime value calculations.

4. Contextual Real-time Features:

   • Incorporates immediate context like current location, time of day, or local events.

   • These features are combined with the user's historical profile for full context.

Hypothetical Example: An LLM-Powered Promotion System inside Grab

Let's walk through a hypothetical scenario of how Grab might use this system for both in-app messaging and general promotions:

Scenario: Grab wants to promote its food delivery service to increase lunch orders on weekdays.

1. Campaign Setup:

   • Marketing team defines the campaign: "Increase weekday lunch orders by 20% over the next month"

   • Target audience: Users who have ordered dinner but rarely lunch

2. User Selection:

   • System identifies 100,000 users matching the criteria

3. Contextual Data Retrieval:

   • For each user, the system retrieves:

     • Past order history (frequency, typical order times, cuisine preferences)

     • Work location (if available)

     • Recent app interactions

4. Prompt Construction:

   • System creates a prompt like: "Generate a lunch promotion message for a user who typically orders dinner. They prefer Thai cuisine and have recently browsed Italian restaurants. Their work is near [specific location]. It's currently Tuesday at 11:30 AM."

5. Message Generation:

   • LLM generates multiple variants, e.g.:

     1. "Craving a midday Thai feast? We've got lunch covered! Order now for 20% off your favorite Thai spots near [work location]."

     2. "Switch up your lunch routine! Enjoy 15% off Italian cuisine, delivered to your office by 12:30. Perfect for a quick workday treat!"

6. Quality Check & A/B Testing:

   • Messages are checked for brand voice and compliance

   • System sets up an A/B test with these two variants

7. Message Delivery:

   • Users receive the promotional message in-app at 11:45 AM

   • Some users might also receive a push notification based on their preferences

8. Real-time Adjustment:

   • User A opens the app at 12:15 PM and browses Italian restaurants

   • System immediately updates their profile and adjusts the promotion:

     • If they haven't seen the message yet, they'll now see the Italian-focused variant

     • If they've seen but not acted on the Thai promotion, a follow-up message might be generated: "How about Italian for a change? Enjoy 15% off your first lunch order from [Popular Italian Restaurant]!"

9. Performance Tracking:

   • System monitors open rates, click-throughs, and orders

   • Data shows the Italian-focused message performing 15% better

   • This insight is fed back into the system for future campaigns

10. Continuous Optimization:

    • Over the campaign period, the LLM fine-tunes its approach based on performance data

    • By week 3, it might generate messages like: "Beat the midweek slump with a delicious lunch delivered to your desk! Your favorite [Cuisine Based on Recent Orders] is just a click away. Enjoy 20% off when you order before 12 PM!"

This example showcases how the LLM-powered system can create highly personalized, context-aware promotions that adapt in real time to user behavior. It demonstrates the system's ability to handle both broad promotional campaigns and individualized in-app messaging, all while continuously learning and optimizing for better performance.

The Real Value of LLMs: Beyond Message Delivery

While it's easy to view LLMs as sophisticated delivery mechanisms for pre-determined promotions, their potential extends far beyond this:

1. Natural Language Generation: LLMs excel at crafting engaging, context-appropriate language that adapts to user preferences and consistently incorporates a brand voice.

2. Context Understanding and Synthesis: They have a unique ability to interpret complex user situations by synthesizing multiple data points, and identifying subtle patterns that might elude human analysts.

3. Dynamic Personalization: LLMs can generate highly personalized content on the fly, adapting messages based on real-time user data and preferences.

4. Creative Ideation: They can assist in generating novel promotion ideas, suggesting creative combinations of offers, and proposing new marketing angles.

5. Multimodal Integration: Advanced LLMs can work with multiple types of data, suggesting appropriate visual elements and crafting promotions suitable for different channels.

6. Rapid Prototyping and A/B Testing: LLMs can quickly generate multiple versions of a promotion, enabling efficient A/B testing at scale and rapid iteration based on performance feedback.

7. Handling Edge Cases: They are particularly adept at adapting to unusual scenarios or user behavior patterns, reducing the need for human intervention in non-standard situations.

8. Continuous Learning and Adaptation: While not learning in the traditional sense, LLMs can be fine-tuned or prompted to adapt to changing market trends and user preferences over time.

Strategic Implications and Considerations

The integration of LLMs into promotional systems has far-reaching strategic implications:

1. Efficiency and Scale: The dramatic reduction in time-to-market for promotions (from 100 hours to 1.5 hours) allows for more agile and responsive marketing strategies.

2. Personalization at Scale: LLMs enable a level of individual tailoring that was previously infeasible, potentially leading to significantly improved engagement rates.

3. Resource Allocation: With LLMs handling much of the content generation and optimization, marketing teams can focus on higher-level strategy and creative direction.

4. Data Strategy: The effectiveness of LLM-powered systems heavily depends on the quality and comprehensiveness of the underlying data. Companies must prioritize robust data collection and management practices.

5. Ethical Considerations: As personalization becomes more sophisticated, companies must be mindful of privacy concerns and ensure transparent and ethical use of user data.

6. Competitive Advantage: Early adopters of advanced LLM-powered promotional systems may gain a significant edge in customer engagement and retention.

Challenges and Limitations

Despite the immense potential, it's important to acknowledge the challenges:

1. Data Quality and Real-time Integration: The system's effectiveness heavily relies on high-quality, real-time data. Implementing robust feature engineering and a real-time feature store is crucial and often more challenging than the LLM integration itself.

2. Model Fine-tuning and Optimization: Effectively fine-tuning LLMs for specific use cases requires substantial data and expertise. Techniques like Direct Preference Optimization (DPO) need careful implementation.

3. Balancing Automation and Human Oversight: While LLMs can automate much of the process, human oversight remains crucial for maintaining brand voice, ensuring ethical practices, and handling complex strategic decisions.

4. Cost-Benefit Analysis: Implementing and maintaining an LLM-powered system can be resource-intensive. Companies must carefully weigh the potential benefits against the costs and complexity.

Conclusion

The potential integration of LLMs into in-app promotional systems represents an exciting development in marketing technology. By reducing time-to-market while enhancing personalization, LLMs could reshape digital marketing. However, success in this domain requires a holistic approach that combines advanced technology with strategic thinking, robust data practices, and a deep understanding of customer needs.

As we move forward, companies that can effectively harness the power of LLMs while navigating the associated challenges will be well-positioned to lead in the era of hyper-personalized, real-time marketing. The future of in-app promotions is likely not just automated – it's hopefully intelligently augmented, creatively enhanced, and deeply personalized.

If you found this piece useful or interesting, don't hesitate to share it with your network.

If this was shared with you and you liked the content, do consider subscribing below to receive the next piece directly.

Disclaimer: These are the personal opinions of the author. Any assumptions, or opinions stated here are theirs and not representative of their employers, current or past.