Greetings, readers and avid listeners and technology enthusiasts! You're either reading or tuned in to the audio-only podcast of The Lindahl Letter, now in its 154th week. Remember, an extra fresh and original edition lands in your inbox every Friday. Now certified with a three year proven track record. Today, we delve into an intriguing theme: “My 2024 Predictions.” Let's explore the future together.1. Generative AI's Expanding Horizons: We're on the brink of witnessing a generative AI leap forward into agency and actions. The upcoming wave is set to introduce more nuanced language models and sophisticated image generators. Imagine a world where content creation and design are revolutionized, and software development is seamlessly intuitive. A standout prediction? Micro-targeting will become a staple, with chat agents offering highly personalized experiences, finely attuned to individual preferences and interests. I think it's actually going to get uncomfortable with how far people are going to take targeting in 2024. 2. The Evolution of AI-Powered Automation: AI's influence in automation is deepening its roots across various sectors. We'll analyze potential milestones in logistics, retail, and online services. Could these innovations redefine job roles and reshape business workflows? Let's ponder the possibilities. I think people are going to jump in and start automating all sorts of things that might deserve automation and others that maybe should have waited for a more mature point in the technology development curve.3. AI Ethics and Legislative Landscape: As AI entwines more with our daily lives, the drumbeat for ethical standards and regulations grows louder. We'll reflect on how various nations might navigate the governance of AI and its impact on global AI development and cooperation. Don’t worry this won’t be a purely legislative capture point of view or a comparative political analysis.4. AI policy may abound: AI's Role in Government and Public Sector is going to increase. AI's potential in enhancing public administration, policy formulation, and citizen services is immense. The discussion will spotlight AI's applications in public safety, urban planning, and social welfare initiatives, marking a significant shift in governmental functions. I think things on this front are going to get moving at a rapid speed in 2024. 5. Quantum Computing Breakthroughs: Turning our gaze to quantum computing, I’ll speculate on its role in tackling problems that are currently beyond classical computing's reach. From cryptography to material science, the implications are vast and profound. Maybe 2024 is the year people use some of that IBM quantum computing and share the results. 6. AI in Healthcare - Next Frontiers: The healthcare sector is ripe for AI-driven innovation. We'll dive into expected advancements in personalized medicine, advanced diagnostics, and AI's emerging role in streamlining healthcare administration and enhancing patient care.What’s next for The Lindahl Letter? * Week 155: Generative AI's Expanding Horizons* Week 156: The Evolution of AI-Powered Automation* Week 157: AI Ethics and Legislative Landscape* Week 158: AI policy may abound* Week 159: Quantum Computing Breakthroughs* Week 160: AI in HealthcareIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you for joining us this week. Stay curious, stay informed, and enjoy the week ahead! This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Please accept this note of gratitude for being a part of this journey. We made it. Some of you have been a part of the entire journey. Three years ago on January 26, 2021, this very Substack started rocking and rolling along a weekly journey to share research notes. My backlog of weekly items to cover is still pretty darn larger. It contains well over 100 blocks of potential writing content. A few larger projects are lurking within that backlog that could be stacked up block by block into future books, manuscripts, or larger articles. I’m feeling reflective today about the nature of and future of Stubstack as a publishing platform. A lot of writers flocked to Substack and it as a platform has certainly helped nurture independent writing. Newsletters have come a long way over the years, but for the most part they are fundamentally the same asymmetric writer to audience communication method. Within the situation we are experiencing at the moment it appears to be Substack as a platform that has changed. I believe and consider it to be true that sunlight has always been the best disinfectant. Understanding begins the path to knowledge. Substack as a platform is at a crossroads. It’s my guess that the platform that is Substack will radically change in 2024. To be honest about that change, I’d have to say I’m not entirely sure what will happen [1]. Generally, I’m going to keep writing and publishing until a move to my core WordPress domain is required [2]. Everything is all set up over at that domain just in case things have to be moved, but I’m legitimately hoping that the Substack community survives the year. My corpus of writing is well over 5 million words and while none of them are particularly spicy or super eventful they were written to be shared. You can tell here now that you made it to the third paragraph that this missive has gone back to my previous writing strategy and is not reduced into highly curated bullet points. That was something that I tried out to see if that modern communication strategy would work for my research notes. I think my preference going forward will be to write in a more long form communication structure. Bullet points have a place and are great for reducing large amounts of information into something more palatable. My writing generally has been more about being a self contained research note that brings forward a degree of understanding about something complex. That will most certainly be the standard going forward. Things that catch my attention are going to receive coverage and that will be the ongoing basis of each weekly Lindahl Letter.Links and thoughts:* I listed to Hard Fork this week “The Times Sues OpenAI + A Debate Over iMessage + Our New Year’s Tech Resolutions” *  I read this paper “Mixtral of Experts” https://arxiv.org/pdf/2401.04088.pdf Footnotes:[1] I read this article from Platformer:[2] https://www.nelslindahl.com/ or here https://www.nelslindahl.com/weblog/ What’s next for The Lindahl Letter? * Week 154: My 2024 predictionsIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Maybe a longer title for this post could be, “Bayesian Models and Elections: A Dive into the Dance of Uncertainty.” This is the 150th transmission of the Lindahl Letter.In the vast and often unpredictable theater of electoral forecasting, the quest for precision is a relentless pursuit. The choreography of voter behavior is a complex ballet, orchestrated by a myriad of factors—societal tremors, economic tides, the charisma of candidates, and the machinations of campaign strategies. Amidst this swirling cauldron of variables, the call for a more nuanced forecasting method is loud and clear. And what answers the call with a finesse born of probabilistic reasoning is the realm of Bayesian models. These statistical marvels stand at the confluence of data and uncertainty, offering a refined lens to dissect the electoral enigma.The essence of Bayesian statistics, a legacy of Thomas Bayes, is a narrative of evolving beliefs in the face of emerging evidence. It's a realm where estimates aren't static, but dynamic, continually reshaped by the rhythm of new data—a narrative that resonates with the pulsating heart of electoral dynamics.In the Bayesian narrative, the tale begins with initial beliefs, our prior probabilities. As the story unfolds with new data—the likelihood—our beliefs morph, culminating in updated beliefs or posterior probabilities. This dance of iterative learning is akin to the dynamism of electoral scenarios, where a single debate, policy announcement, or campaign rally could tilt the scales of public sentiment.A compelling act in the Bayesian play is its ability to weave historical election data into the forecasting fabric. It’s not just about the now, but a dialogue with the past, understanding how the ghost of incumbency, the whisper of economic indicators, or the shout of demographic shifts have choreographed electoral outcomes before.And then, there’s the magnum opus of Bayesian models—the articulation of uncertainty. Unlike the static snapshot often rendered by traditional polling, Bayesian models compose a symphony of probability distributions. They unveil a spectrum of possible electoral outcomes, each with its associated probability, painting a picture of electoral reality that's as rich as it is realistic.The spotlight often falls on case studies like the 2012 and 2016 U.S. Presidential Elections, where the Bayesian choreography, as orchestrated by platforms like Nate Silver’s FiveThirtyEight, navigated the electoral tumult with a commendable degree of accuracy. By embracing uncertainties and dancing with historical context, Bayesian models orchestrate a forecast that traditional polling methods seldom match.Yet, the narrative isn’t without its share of cliffhangers. The hurdles of data scarcity, model misspecification, and computational intricacies are challenges that beckon solutions. Despite these, the Bayesian voyage into electoral forecasting holds a promise—of rendering narratives that are not only statistically sound but resonate with intuitive clarity.As the electoral saga continues to unfold, the allure for better forecasting tools is a relentless whisper. Bayesian models, with their eloquence in narrating the dance of uncertainty, emerge as potent companions for pollsters and policymakers. They underline an electoral truism—in a realm replete with uncertainties, understanding and embracing these uncertainties isn’t just the hallmark of wisdom, but a cornerstone of robust electoral forecasting.A few scholarly articles I found interesting this week:Linzer, D. A. (2013). Dynamic Bayesian forecasting of presidential elections in the states. Journal of the American Statistical Association, 108(501), 124-134. https://www.ocf.berkeley.edu/~vsheu/Midterm%202%20Project%20Files/Linzer-prespoll-May12.pdf Lock, K., & Gelman, A. (2010). Bayesian combination of state polls and election forecasts. Political Analysis, 18(3), 337-348. https://academiccommons.columbia.edu/doi/10.7916/D88K7GV1/download Heidemanns, M., Gelman, A., & Morris, G. E. (2020). An updated dynamic Bayesian forecasting model for the US presidential election. Harvard Data Science Review, 2(4), 10-1162. https://assets.pubpub.org/wbec6d9k/9dfc3335-6d48-4f8e-bf5d-0011c7817a09.pdf Olsson, H., Bruine de Bruin, W., Galesic, M., & Prelec, D. (2021). Election polling is not dead: a Bayesian bootstrap method yields accurate forecasts. Preprint at https://osf.io/nqcgs/ If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Title: The Confluence of Agent Systems and Expert Opinion in Election SimulationsIn the ever-evolving landscape of political science and technology, curiosity often paves the way for innovative approaches and fresh perspectives. Recently, a wave of curiosity has washed over me, primarily centered around the exploration of diverse agent systems to simulate elections. This intersection of technology and electoral processes opens up new realms of possibilities, allowing us to mimic, analyze, and potentially enhance our understanding of elections in a simulated environment.Agent systems provide a powerful tool for creating algorithmic or model-based simulations. These systems can be meticulously crafted, incorporating synthetic focus groups and panels that emulate real-world election scenarios. The meticulous design allows for the creation of adversarial agents that can engage in debates or various activities that accurately mirror the complexities and dynamics of an actual election. However, my curiosity doesn’t end here. I’m also deeply intrigued by the fusion of election simulation with expert opinion systems. By appending the term ‘systems’ to ‘expert opinion’, the concept transcends beyond individual viewpoints, fostering an environment where aggregated expert opinions are diligently worked upon and analyzed. These collected data become a potent resource, providing invaluable insights that can be seamlessly integrated into the simulated election models.Imagine the immense potential unlocked by the combination of these two realms. The simulated agents, fortified with synthesized expert opinions, could operate in a nuanced manner that echoes the depth and diversity of actual election contenders and voters. These enhanced agents could engage in debates, make decisions, and navigate the election simulation with a level of sophistication that brings us closer to understanding the myriad factors influencing election outcomes.Through this amalgamation, the simulation becomes a crucible where technological prowess meets the wisdom of expertise. The interplay between algorithmically driven simulations and the rich reservoir of expert opinions can unveil unprecedented avenues for exploring electoral processes. It can deepen our comprehension, offering a clearer lens through which we may view the multifaceted realms of elections.In conclusion, the fusion of various agent systems with expert opinion systems presents a promising frontier in the world of election simulations. By harnessing the collective wisdom of experts and embedding this knowledge within algorithmic agents, we stand on the brink of developing more nuanced, realistic, and insightful election simulation models. The curiosity driving this exploration is not merely a personal quest, but rather a shared journey towards enriching our understanding and approaches to simulating and analyzing electoral processes.What’s next for The Lindahl Letter? * Week 147: Bayesian Models* Week 148: Running Auto-GPT on election models* Week 149: Sentiment Analysis* Week 150: Voter ModelsIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Let’s get to work unpacking the Delphi Method & Door-to-Door Canvassing. This is the Substack deep dive you don’t want to miss.Hey there, dear weekly Substack readers!Today, let's embark on a journey through two fascinating potentially election related realms: the Delphi Method and Door-to-Door Canvassing. While these two concepts might seem worlds apart, there is a curious intersection between the two of them that's worth exploring. So, grab your favorite beverage, and let's dive in! I had two shots of espresso and they were delightful. Delphi Method: More Than Just Ancient GreeceFirst up, the Delphi Method. No, we're not time-traveling back to ancient Greece using the world’s finest Delorean, but we are diving into a method inspired by the oracle of Delphi. It's a structured communication technique designed for interactive forecasting. Picture this: a group of experts, multiple rounds of questionnaires, and a quest for consensus. The beauty of this method? It taps into collective intelligence while ensuring every expert voice gets its moment in the sun, minus the overshadowing by dominant personalities. Generally this method is unlikely to occur naturally today due in part to the overwhelming decay in civility that has occurred. People just don’t cross over into different parisian camps these days. Something has distinctly changed in our politics. Prokesch, T., Von der Gracht, H. A., & Wohlenberg, H. (2015). Integrating prediction market and Delphi methodology into a foresight support system—Insights from an online game. Technological Forecasting and Social Change, 97, 47-64 [1]. Dalkey, N. C., Brown, B. B., & Cochran, S. (1969). The Delphi method: An experimental study of group opinion (Vol. 3, p. 107). Santa Monica, CA: Rand Corporation [2].Door-to-Door Canvassing: Old School politics, But otherwise pure GoldSwitching gears, let's talk about the age-old art of door-to-door canvassing. It's personal, it's direct, and it's all about that face-to-face interaction. Whether it's political volunteers rallying support or grassroots movements gathering opinions, this method has stood the test of time. Why? Because nothing beats the authenticity of a real conversation. Sure people are less likely to want to answer the door or talk politics at the front door, but this method does still show signs of working.Green, D. P., Gerber, A. S., & Nickerson, D. W. (2003). Getting out the vote in local elections: Results from six door-to-door canvassing experiments. The Journal of Politics, 65(4), 1083-1096 [3]. The Unexpected CrossoverNow, for the fun part. How do these two methods intertwine?Imagine harnessing the Delphi Method's expert-driven insights to supercharge a door-to-door canvassing campaign. Before our canvassers even lace up their shoes, we could have a panel of experts—from veteran canvassers to communication gurus—forecasting the best strategies, pinpointing challenges, and highlighting golden opportunities.That type of Magic could happenMarrying the Delphi Method's structured insights with the grassroots authenticity of door-to-door canvassing could:* Elevate the Message: Crafting narratives that truly resonate.* Stay Two Steps Ahead: Predicting and preparing for potential challenges.* Strategize Like a Pro: Directing efforts where they count the most.Wrapping UpMerging the old with the new, the traditional with the innovative, can lead to some unexpected and powerful synergies. And isn't that what we're all about here on Substack? Exploring, questioning, and connecting the dots in unexpected ways.Stay curious, and until next time!Dr. Nels LindahlFootnotes:[1] https://www.researchgate.net/profile/Heiko-Von-Der-Gracht/publication/260755254_Integrating_prediction_market_and_Delphi_methodology_into_a_foresight_support_system_-_Insights_from_an_online_game/links/5c339e35299bf12be3b5592a/Integrating-prediction-market-and-Delphi-methodology-into-a-foresight-support-system-Insights-from-an-online-game.pdf [2] https://apps.dtic.mil/sti/trecms/pdf/AD0690498.pdf [3] https://d1wqtxts1xzle7.cloudfront.net/45996527/Getting_Out_the_Vote_in_Local_Elections_20160527-16511-1wf5rrd-libre.pdf?1464362918=&response-content-disposition=inline%3B+filename%3DGetting_Out_the_Vote_in_Local_Elections.pdf&Expires=1696684199&Signature=S6S9UNmWYRMepopnbBWQlGkCn4q4C889yqi3aoE~-47Z~DL2Hpw5TWKDz6Cq4IF9~gp-sfEPaVehWkrW7YiYQLLL0f6XEsNNtlU3WUl4NSee2JH1B2CNTWcy9glqPjVo6KBfe6oKUYr4YlCatCXgDgJEL~HtRsIiwswn4XxGpWAv~7sLT-X5M8Zc13wVlYl8MEzNF32WpOM5JaJUtUA8Z5k8G2cMgHWzRRYyB6GXf1Pr2MWovSCameHEHC~G44wcCYoK-54jdYdnP605msL6gifKpj0dp58ETNOcFnBvCVwU5hjfUcIgjCJLpDTlXV7OLokXfL3uydrisDoN~H3QZA__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Don’t panic, the Google Scholar searches are coming in fast and furious on this one [1]. We had a footnote in the first sentence today. Megan Tomlin writing over at neo4j had probably the best one line definition of the difference by noting that knowledge graphs are going to be in the human readable data camp and vector databases are more of a black box [2]. I actually think that eventually one super large knowledge graph will emerge and be the underpinning of all of this, but that has not happened yet given that the largest one in existence Google holds will always remain proprietary. Combining two LLMs… right now you could call them one after another, but I’m not finding an easy way to pool them into a single model. I wanted to just say to my computer, “use Baysian pooling to combine the most popular LLMs from Hugging Face,” but yeah that is not an available command at the moment. A lot of incompatible content is being generated in the vector database space. People are stacking LLMs and working in sequence or making parallel calls to multiple-models. What I was very curious about was how to go about the process of merging LLMs, combining LLMs, actual model merges, ingestion of models, or even a method to merge transformers. I know that is a tall order, but it is one that would take so much already spent computing cost and move it from sunk to additive in terms of value. A few papers exist on this, but they are not exactly solutions to this problem. Jiang, D., Ren, X., & Lin, B. Y. (2023). LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion. arXiv preprint arXiv:2306.02561. https://arxiv.org/pdf/2306.02561.pdf  you can see more content related to this one here https://yuchenlin.xyz/LLM-Blender/Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., ... & Wang, C. (2023). AutoGen: Enabling next-gen LLM applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155. https://arxiv.org/pdf/2308.08155.pdf Chan, C. M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., ... & Liu, Z. (2023). Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201. https://arxiv.org/pdf/2308.07201.pdf Most of the academic discussions and even the cutting edge papers like AutoGen are about orchestration of models instead of merging, combining, or ingestion of many models into one. I did find a discussion on Reddit from earlier this year about how to merge the weights of transformers [3]. It’s interesting what things end up on reddit. Sadly that subreddit is closed due to a dispute over 3rd party plugins. Exploration into merging and combining Large Language Models (LLMs) is indeed at the frontier of machine learning research. While academic papers like "LLM-Blender" and "AutoGen" offer different perspectives, they primarily focus on ensembling and orchestration rather than true model merging or ingestion. The challenge lies in the inherent complexities and potential incompatibilities when attempting to merge these highly sophisticated models.The quest for effectively pooling LLMs into a single model or merging transformers is a journey intertwined with both theoretical and practical challenges. Bridging the gap between the human-readable data realm of knowledge graphs and the more opaque vector database space, as outlined in the beginning of this podcast, highlights the broader context in which these challenges reside. It also underscores the necessity for a multidisciplinary approach, engaging both academic researchers and the online tech community, to advance the state of the art in this domain.In the upcoming weeks, we will delve deeper into the community-driven solutions, and explore the potential of open-source projects in advancing the model merging discourse. Stay tuned to The Lindahl Letter for a thorough exploration of these engaging topics.Footnotes:[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=knowledge+graph+vector+database&btnG= [2] https://neo4j.com/blog/knowledge-graph-vs-vectordb-for-retrieval-augmented-generation/ [3] https://www.reddit.com/r/MachineLearning/comments/122fj05/is_it_possible_to_merge_transformers_d/ What’s next for The Lindahl Letter? * Week 145: Delphi method & Door-to-door canvassing* Week 146: Election simulations & Expert opinions* Week 147: Bayesian Models* Week 148: Running Auto-GPT on election models* Week 149: Modern Sentiment AnalysisIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

After the adventures of last week, I started this writing adventure wanting to try to figure out what people were doing with LangChain and social media. People are both generating content for social media using LLMs and oddly enough repurposing content as well. We have to zoom out for just a second and consider the broader ecosystem of content. In the before-times, people who wanted to astroturf or content farm had some work to do within the content creation space. Now ChatGPT has opened the door and let the power of synthetic content creation loose. You can create personas and just have them generate endless streams of content. People can even download and run models trained for this purpose. It’s something I’m legitimately worried about for this next election cycle. Sometimes I wonder how much content within the modern social media spaces is created artificially. Measuring that is actually pretty difficult. It’s not like organically created content gets a special badge or recognition. For those of you who were interested in finding out insights on any topic with a plugin that works with the OpenAI ChatGPT system then you could take a moment and install “The Yabble ChatGPT Plugin” [1]. Fair warning on this one I had to reduce my 3 plugins down to just Yabble and be pretty explicit in the prompts within ChatGPT to make it do some work. Sadly, I could not just login to Yabble and had to book a demo with them to get access. Stay tuned on that one to get more information on how that system works. I had started by searching out plugins to have ChatGPT analyze social media. This has become easier now with the announcements that OpenAI can openly use Bing search [2]. Outside of searching using any OpenAI tooling like ChatGPT, Google was pretty clear on the reality that what I was really looking for happened to actually be marketing tools. Yeah, I went down the SEO Assistant rabbit hole and it was shocking. So much content exists in this space that is like watching a very full ant farm for the most part. Figuring out where to jump in without getting scammed is probably a questionable decision framework. Whole websites and ecosystems could be synthetically generated pretty quickly. It’s not exactly one click turn key deployments, but it is getting close to that level of content farming.I was willing to make the assumption that people who were going to the trouble of making actual plugins for ChatGPT within the OpenAI platform are probably going to be more interesting and maybe are building actual tooling. For those of you who are using ChatGPT with OpenAI and have the plus subscription you just have to open a new chat, expand the plugin area, and scroll down to the plugin store to search for new ones…I also did some searches for marketing tools. I’m still struck with the possibility that a lot of content is being created and marketed to people. It’s not the potential flooding of content that becomes so overwhelming that nobody is able to navigate the internet anymore. We are getting very close to the point where it would be entirely possible for the flooding of new content to occur in ways that simply overwhelm everybody and everything with new content. This would be like the explosion of ML/AI papers over the last 5 years, but maybe 10x or 100x even that digital content boom [3].Footnotes:[1] https://www.yabble.com/chatgpt-plugin[2] https://www.reuters.com/technology/openai-says-chatgpt-can-now-browse-internet-2023-09-27/ [3] https://towardsdatascience.com/neurips-conference-historical-data-analysis-e45f7641d232 What’s next for The Lindahl Letter? * Week 144: Knowledge graphs vs. vector databases* Week 145: Delphi method & Door-to-door canvassing* Week 146: Election simulations & Expert opinions* Week 147: Bayesian Models* Week 148: Running Auto-GPT on election modelsIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

And now it’s time to pivot toward the “Learning LangChain” topic…The most fun introduction to LangChain seems to be from DeepLearning.ai with Andrew Ng and Harrison Chase [1]. You can expect to spend a couple of hours to complete the process of watching the videos and absorbing the content. Make sure you use a browser window large enough to support both the jupyter notebook and the video. You are probably going to want these items to run side by side. This course covers models, prompts, parsers, memory, chains, and agents. The part of this learning package that I was the most interested in learning more about was how people are using agents and of course what sort of plugins could that yield as use cases in the generative AI space. Going forward I think agency will be the defining characteristic of the great generative AI adventure. These applications are going to do things for you and some of those use cases are going to be extremely powerful. After that course I wanted to dig in more and decided to go ahead and learn everything I could from the LangChain AI Handbook [2]. This handbook has 6 or 7 chapters depending on how you count things. My favorite part about this learning build is that they are using Colab notebooks for hands-on development during the course of the learning adventure. That is awesome and really lets you get going quickly. A side quest spawned out of that handbook learning which involved starting to use Pinecone in general which was interesting. You can do a lot with the Pinecone including building AI agents and chatbots. I’m going to spend some time working on the udemy course “Develop LLM powered applications with LangChain” later this weekend [3]. You can also find a ton of useful information within the documentation for LangChain including a lot of content about agents [4].You might now be wondering what alternatives to LangChain exist… I started looking around at AutoChain [5], Auto-GPT [6], AgentGPT [7], BabyAGI [8], LangDock [9], GradientJ [10], Flowise AI [11], and LlamaIndex [12]. Maybe you could also consider TensorFlow to be an alternative. You can tell from the combination of companies and frameworks being built out here a lot of attention is on the space between LLMs and taking action. Getting to the point of agency or taking action is where these spaces are gaining and maintaining value. Footnotes:[1] https://learn.deeplearning.ai/langchain/lesson/1/introduction [2] https://www.pinecone.io/learn/series/langchain/ [3] https://www.udemy.com/course/langchain/[4] https://python.langchain.com/docs/modules/agents/ [5] https://github.com/Forethought-Technologies/AutoChain [6] https://github.com/Significant-Gravitas/Auto-GPT[7] https://github.com/reworkd/AgentGPT[8] https://github.com/miurla/babyagi-ui [9] https://www.langdock.com/ [10] https://gradientj.com/[11] https://flowiseai.com/[12] https://www.llamaindex.ai/What’s next for The Lindahl Letter? * Week 143: Social media analysis* Week 144: Knowledge graphs vs. vector databases* Week 145: Delphi method & Door-to-door canvassing* Week 146: Election simulations & Expert opinions* Week 147: Bayesian ModelsIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

You can feel the winds of change blowing and the potential of people building out election expert opinion chatbots. Maybe you want to know what they are probably going to use to underpin that sort of effort. If you were going out to build some generative AI chatbots for you might very well use one of the 5 systems we are going to dig into today.* Voiceflow - This system may very well be the most prominent of the quick to market AI agent building platforms [1]. I have chatbots deployed to both Civic Honors and my main weblog powered by Voiceflow.* LangFlow - You are going to need to join the waitlist for this one to get going [2]. I’m still on the waitlist for this one… * Botpress - Like Voiceflow this system lets you pretty quickly jump into the building process of actual chatbot workflows [3]. To be fair with this one I was not able to build and deploy something into production within minutes, but you could do it pretty darn quickly if you had a sense of what you were trying to accomplish. I built something on Botpress and it was pretty easy to use. After login I clicked answer questions from websites to create a bot. I added both Civic Honors and my main Nels Lindahl domain. They just jumped in and advised me that the knowledge upload was complete. Publishing the bot is not as low friction as the Voiceflow embedding launch point, but it was not super hard to work with after you find the share button.* FloWiseAI - You will find this is the first system on the list that will require you to get out of your web browser, stretch a bit, and open the command line to get this one installed with a rather simple “npm install -g flowise” command [4]. I watched some YouTube videos on how to install this one and it almost got me to flip over into Ubuntu Studio. Instead of switching operating systems I elected to just follow the regular Windows installation steps.* Stack AI - With this one you are right back into the browser and you are going to see a lot of options to start building new projects [5].All of these chatbots built using a variety of generative AI models are generally working within the same theory of building. The conversation is being crafted with a user and some type of exchange with a knowledge base. For the most part the underlying LLM is being used to facilitate the conversational part of the equation while some type of knowledge base is being used to gate, control, and drive the conversation based on something deeper than what the LLM would output alone. It’s an interesting building technique and one that would not have been possible just a couple of years ago, but the times have changed and here we are in this brave new world where people can build, deploy, and be running a generative AI chatbot in a few minutes. It requires some planning about what is being built, you need some type of knowledgebase, and the willingness to learn the building parameters. None of that is a very high bar to pass. This is a low friction and somewhat high reward space for creating conversational interactions. Messing around with all these different chatbot development systems made me think a little bit more about how LangChain is being used and what the underlying technology is ultimately capable of facilitating [6]. To that end I signed up for the LangSmith beta they are building [7]. Sadly enough “LangSmith is still in closed beta” so I’m waiting on access to that one as well. During the course of this last week I have been learning more and more about how to build and deploy chatbots that take advantage of LLMs and other generative AI technologies. I’m pretty sure that the development of agency to machine learn models is going to strap rocket boosters to the next stage of technological deployment. Maybe you are thinking that is hyperbole… don’t worry or panic, but you are very soon going to be able to ask these agents to do something and they will be able to execute more and more complex actions. That is the essence of agency within the deployment of these chatbots. It’s a very big deal in terms of people doing basic task automation and it may very well introduce a distinct change to how business is conducted by radically increasing productivity. Footnotes:[1] https://www.voiceflow.com/ [2] https://www.langflow.org/ [3] https://botpress.com/ [4] https://flowiseai.com/ [5] https://www.stack-ai.com/ [6] https://www.langchain.com/ [7] https://www.langchain.com/langsmith What’s next for The Lindahl Letter? * Week 142: Learning LangChain* Week 143: Social media analysis* Week 144: Knowledge graphs vs. vector databases* Week 145: Delphi method & door to door canvasing* Week 146: Election simulationsIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

Sometimes a simplified model of something is easier to work with. We dug into econometric models recently during week 136 and they can introduce a high degree of complexity. Even within the world of econometrics you can find information about proxy models. In this case today we are digging into proxy models for elections. My search was rather direct. I was looking for a list of proxy models being used for elections [1]. I was trying to dig into election forecasting proxy models or maybe even some basic two step models. I even zoomed in a bit to see if I could get targeted on machine learning election proxy models [2].After a little bit of searching around it seemed like a good idea to maybe consider what it takes to generate a proxy model equation to represent something. Earlier I had considered what the chalk model of election prediction would look like with using a simplified proxy of voter registration as an analog for voting prediction. I had really thought that would end up being a highly workable proxy, but it was not wholesale accurate. Here are 3 papers I looked at this week:Hare, C., & Kutsuris, M. (2022). Measuring swing voters with a supervised machine learning ensemble. Political Analysis, 1-17. https://www.cambridge.org/core/services/aop-cambridge-core/content/view/145B1D6B0B2877FC454FBF446F9F1032/S1047198722000249a.pdf/measuring_swing_voters_with_a_supervised_machine_learning_ensemble.pdf Zhou, Z., Serafino, M., Cohan, L., Caldarelli, G., & Makse, H. A. (2021). Why polls fail to predict elections. Journal of Big Data, 8(1), 1-28. https://link.springer.com/article/10.1186/s40537-021-00525-8 Jaidka, K., Ahmed, S., Skoric, M., & Hilbert, M. (2019). Predicting elections from social media: a three-country, three-method comparative study. Asian Journal of Communication, 29(3), 252-273. http://www.cse.griet.ac.in/pdfs/journals20-21/SC17.pdf I spent some time messing around with OpenAI’s GPT-4 on this topic. That effort drove down to a few proxy models that are typically used. The top 10 seemed to be the following: social media analysis, google trends, economic indicators, fundraising data, endorsement counts, voter registration data, early voting data, historical voting patterns, event-driven, and environmental factors. Combining all 10 proxy models into a single equation would result in a complex, multi-variable model. Here's a simplified representation of such a model:E=α1​(S)+α2​(G)+α3​(Ec)+α4​(F)+α5​(En)+α6​(VR)+α7​(EV)+α8​(H)+α9​(Ed)+α10​(Ef)+βWhere:* E is the predicted election outcome.* α1, α2​,...α10 are coefficients that determine the weight or importance of each proxy model. These coefficients would be determined through regression analysis or other statistical methods based on historical data.* S represents social media analysis.* G represents Google Trends data.* Ec represents economic indicators.* F represents fundraising data.* En represents endorsement count.* VR represents voter registration data.* EV represents early voting data.* H represents historical voting patterns.* Ed represents event-driven models.* Ef represents environmental factors.* β is a constant term.This equation is a linear combination of the proxy models, but in reality, the relationship might be non-linear, interactive, or hierarchical. The coefficients would need to be determined empirically, and the model would need to be validated with out-of-sample data to ensure its predictive accuracy. Additionally, the model might need to be adjusted for specific elections, regions, or time periods. It would be interesting to try to pull together the data to test that type of complex multivariable model. Maybe later on we can create a model with some agency designed to complete that task.Footnotes:[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+proxy+models&btnG=[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+proxy+models+machine+learning&btnG=What’s next for The Lindahl Letter? * Week 141: Building generative AI chatbots* Week 142: Learning LangChain* Week 143: Social media analysis* Week 144: Knowledge graphs vs. vector databases* Week 145: Delphi methodIf you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com