Show notes
Yue Meng is the Senior Data Scientist at Delivery Hero, an on-demand delivery platform. Previously she was a marketing consultant for Ninah Consulting. She has a master’s degree in statistics. She is based in Berlin, Germany. Questions Yue Answered in this Episode:What methodologies do you use to evaluate your various campaigns? What are your plans for A/B testing once device IDs are no longer available with the data privacy changes to iOS14? Can you give us an example of how you would analyze an offline campaign?How do you know if your baseline model is accurate enough?How would you improve your model if it wasn’t accurate enough?Can you use the control group only to focus testing? Timestamp:1:02 Yue’s background4:45 Why Delivery Hero trusts A/B tests7:54 Plans for marketing campaign testing without device IDs11:29 KPIs for evaluating offline campaigns15:10 Measuring uplift from offline campaigns with time series21:39 Calculating accuracy with baseline models24:30 Using geo as a proxy for user-level data31:34 Looking ahead at testing solutionsQuotes: (9:21-9:43) “This offline scenario is very similar to an online scenario without the device ID, I would say. So probably some models that we’re using for the offline campaigns estimation can be used on the online campaigns also. That’s what we are currently trying to do.”(17:07-17:20) “By using the time series analysis, the simplest model you can use, you’re not adding any other factors or any other features. You only focus on the time series by itself.”(24:30-24:41) “Now the control group on a user level is impossible. What we can try to do as a proxy is to use instead of user-level to use geo-level, for example, the city level.” Mentioned in this Episode:Yue Meng’s LinkedInDelivery Hero
