Philadelphia Analytics (PhillyAnalytics)

We’re back! for the September Philly Analytics + DataPhilly meetup! 🎉
We’ll explore how AI is reshaping industries—from deploying LLMs and computer vision to transform customer experience, to building rigorous evaluation practices that ensure trust and responsibility in AI adoption.
Our Host and Sponsor, BrainDo is a Philadelphia-based team of digital marketing experts who partner with clients to create meaningful experiences with data-driven results. They host and manage Philly Analytics, including the monthly Web Analytics Wednesdays community meetups.
🔗[ phillyanalytics.org](phillyanalytics.org) | https://www.meetup.com/phillyanalytics/

🎤 Speakers
1️⃣ Lou Zhang, https://apclypsr.github.io/ – Using LLMs and Computer Vision to Accelerate the Customer Experience
Abstract: LLMs and chatbots are the hot thing right now, but how do they actually get implemented, and how does the sausage get made? In this talk, we'll go through how we leveraged AWS and Claude to deliver a chatbot to help customers perform analytics on very large datasets. We'll go through the process of how we transitioned legacy Shiny dashboards, with piecemeal analyses and graphs, to a streamlined interface with Amazon Q where customers can run scenarios and do Q&A on top of the data. The business impact of this technology was significant, allowing customers to drill down deeply into their data, and resulted in a premium tier of pricing for our customers.
In the second part of our talk, we'll go over how we combined technologies in computer vision and transformers to reduce customer onboarding time by 80%. We fine tuned a Hugging Face model, combined with open-source segmentation methods to allow customers to simply snap a picture of their assets to register them instead of manually inputting every piece of data. The model achieves 85% accuracy, which outperforms all previous models for this domain.
Bio: Lou is the Senior Lead of Data Science at Datacor, where he works on projects related to LLMs/chatbots, computer vision, and machine learning for the manufacturing industry. Lou has 10+ years of experience in industrials, with a focus on discrete and process manufacturing. He has experience building data science teams from the ground up, creating products which drive top-line revenue, and conducting patented research and publishing peer-reviewed papers in top journals. His research has been cited and used in far reaching industries, from fisheries to climatology.

2️⃣ Dr. Dhagash Mehta, Google Scholar – How to Choose a Threshold for an Evaluation Metric for Large Language Models
Abstract: To ensure and monitor large language models (LLMs) reliably, various evaluation metrics have been proposed in the literature. However, there is little research on prescribing a methodology to identify a robust threshold on these metrics even though there are many serious implications of an incorrect choice of the thresholds during deployment of the LLMs. Translating the traditional model risk management (MRM) guidelines within regulated industries such as the financial industry, in this talk, I will discuss a step-by-step recipe for picking a threshold for a given LLM evaluation metric. I will emphasize that such a methodology should start with identifying the risks of the LLM application under consideration and risk tolerance of the stakeholders. We then propose concrete and statistically rigorous procedures to determine a threshold for the given LLM evaluation metric using available ground-truth data.

Bio: Dr. Mehta is the Head of Applied AI and Responsible AI Research (Investment Management) at Blackrock Inc. and an Editorial Board Member at the Journal of Financial Data Science and Journal of ESG and Impact Investing. He was also General Chair for the ACM International Conference on AI in Finance 2024 (ICAIF24). Previously he was a Senior Manager,Investment Strategist (Machine Learning – Asset Allocation) at The Vanguard Group, a Senior Research Scientist at United Technologies (UTX) Research Center, a Research Professor at University of Notre Dame, a Fields Institute Postdoc Fellow, a Visiting Fellow at Simons Institute for Theory of Computing at Berkeley, a postdoctoral researcher at the University of Cambridge, Syracuse University (USA) and National University of Ireland Maynooth (Ireland) and a research student at Imperial College London (the UK), the University of Adelaide (Australia).

🗓️ Agenda
6:00 – 6:30 PM: Arrival, networking, and refreshments
6:30 – 7:10 PM: Talk 1 – Lou Zhang
7:10 – 7:50 PM: Talk 2 – Dr. Dhagash Mehta
7:50 – 8:30 PM: Networking
👉 RSVP now to secure your spot and join us for an evening of cutting-edge AI insights, great conversations, and community connections!
Please note, by RSVPing this event you agree to our Code of Conduct