class: center, middle, inverse, title-slide .title[ # Augmented AI for Quality Control and Analytics Education ] .subtitle[ ## Practical Tools, Lessons, and Opportunities ] .author[ ### <br><br>Ying-Ju Tessa Chen, PhD <br><a href="https://scholar.google.com/citations?user=nfXnYKcAAAAJ&hl=en&oi=ao"><svg viewBox="0 0 488 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M488 261.8C488 403.3 391.1 504 248 504 110.8 504 0 393.2 0 256S110.8 8 248 8c66.8 0 123 24.5 166.3 64.9l-67.5 64.9C258.5 52.6 94.3 116.6 94.3 256c0 86.5 69.1 156.6 153.7 156.6 98.2 0 135-70.4 140.8-106.9H248v-85.3h236.1c2.3 12.7 3.9 24.9 3.9 41.4z"></path></svg> Scholar</a>   |   <a href="https://github.com/ying-ju"><svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> <span class="citation">@ying-ju</span></a>   |   <a href="mailto:ychen4@udayton.edu"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"></path></svg> ychen4@udayton.edu</a></br> <br><br><br> <u><b><font color="white">Joint work with:</b></u><br> Fadel M. Megahed, Allison Jones-Farmer, Josh Ferris, Gabe Lee, Brooke Wang   <a href="https://miamioh.edu/fsb/directory/?up=/directory/farmerl2"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> Miami University</a><br> Bianca Maria Colosimo, Marco L.G. Grasso   <a href="https://www.mecc.polimi.it/en/staff/"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> Politecnico di Milano</a><br> Sven Knoth   <a href="https://www.hsu-hh.de/compstat/en/sven-knoth-2"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> Helmut-Schmidt-Universität</a><br/> Douglas C. Montgomery   <a href="https://search.asu.edu/profile/10123"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> Arizona State University</a><br> Hongyue Sun   <a href="https://engineering.uga.edu/team_member/hongyue-sun/"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> University of Georgia</a><br> Inez Zwetsloot   <a href="https://www.uva.nl/en/profile/z/w/i.m.zwetsloot/i.m.zwetsloot.html"><svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;fill:white;" xmlns="http://www.w3.org/2000/svg"> <path d="M326.612 185.391c59.747 59.809 58.927 155.698.36 214.59-.11.12-.24.25-.36.37l-67.2 67.2c-59.27 59.27-155.699 59.262-214.96 0-59.27-59.26-59.27-155.7 0-214.96l37.106-37.106c9.84-9.84 26.786-3.3 27.294 10.606.648 17.722 3.826 35.527 9.69 52.721 1.986 5.822.567 12.262-3.783 16.612l-13.087 13.087c-28.026 28.026-28.905 73.66-1.155 101.96 28.024 28.579 74.086 28.749 102.325.51l67.2-67.19c28.191-28.191 28.073-73.757 0-101.83-3.701-3.694-7.429-6.564-10.341-8.569a16.037 16.037 0 0 1-6.947-12.606c-.396-10.567 3.348-21.456 11.698-29.806l21.054-21.055c5.521-5.521 14.182-6.199 20.584-1.731a152.482 152.482 0 0 1 20.522 17.197zM467.547 44.449c-59.261-59.262-155.69-59.27-214.96 0l-67.2 67.2c-.12.12-.25.25-.36.37-58.566 58.892-59.387 154.781.36 214.59a152.454 152.454 0 0 0 20.521 17.196c6.402 4.468 15.064 3.789 20.584-1.731l21.054-21.055c8.35-8.35 12.094-19.239 11.698-29.806a16.037 16.037 0 0 0-6.947-12.606c-2.912-2.005-6.64-4.875-10.341-8.569-28.073-28.073-28.191-73.639 0-101.83l67.2-67.19c28.239-28.239 74.3-28.069 102.325.51 27.75 28.3 26.872 73.934-1.155 101.96l-13.087 13.087c-4.35 4.35-5.769 10.79-3.783 16.612 5.864 17.194 9.042 34.999 9.69 52.721.509 13.906 17.454 20.446 27.294 10.606l37.106-37.106c59.271-59.259 59.271-155.699.001-214.959z"></path></svg> University of Amsterdam</a><br/><br/> ] .date[ ### October 30, 2025 | Flyer Scholars in Action | University of Dayton ] --- ## The Road to Large Language Models <br> <img src="figs/generative_ai_chart.png" alt="From big data to big models, a flow chart documenting how we got to large language models" width="100%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Comment:** You have been hearing about **big data** in Quality Control for over a decade now. In fact, we presented our paper, [Statistical Perspectives on Big Data](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=ab40f392e653b7336cbebf7c4fb95d3988748282), almost exactly 12 years ago in the ISQC Workshop in Sydney. We now have models that can digest questions/prompts and generate answers based on more than 45TB of text. ] --- ## Uniqueness of LLMs vs. Earlier AI Models .content-box-gray[ .bold[.red[LLMs:]] .bold[The objective is to generate new content rather than analyze existing data.] ] .font90[ - The generated content is based on a .bold[.red[stochastic behavior embedded in generative AI models such that the same input prompts results in different content]]. - LLMs with known model sizes can have up to **540 billion parameters** ([PaLM](https://arxiv.org/abs/2204.02311)). Note that state-of-the-art models like *GPT-o3*, and *Claude Sonnet 3.7* have **not revealed their model sizes**. - With the increase in model size, researchers have observed the **“emergent abilities”** of LLMs, which were **not explicitly encoded in the training**. [Examples include](https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html): + Multi-step arithmetic, and + taking college-level exams. - LLMs are **foundation models** (see [Bommasani et al. 2021](https://arxiv.org/abs/2108.07258)), large pre-trained AI systems that can be **repurposed with minimal effort across numerous domains and diverse tasks.** ] --- ## Beyond the Hype > Megahed, F. M., Chen, Y. J., Ferris, J. A., Knoth, S., & Jones-Farmer, L. A. (2024). How generative AI models such as ChatGPT can be (mis) used in SPC practice, education, and research? An exploratory study. Quality Engineering, 36(2), 287-315. **The question isn't hype - it's how to use these tools responsibly.** - Megahed, F. M., Chen, Y. J., Jones-Farmer, L. A., Lee, Y., Wang, J. B., & Zwetsloot, I. M. (2025). Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applications. arXiv preprint arXiv:2505.14918. - Megahed, F. M., Chen, Y. J., Ferris, J. A., Resatar, C., Ross, K., Lee, Y., & Jones-Farmer, L. A. (2024). ChatISA: A Prompt-Engineered, In-House Multi-Modal Generative AI Chatbot for Information Systems Education. arXiv preprint arXiv:2407.15010. - Megahed, F. M., Chen, Y. J., Zwetsloot, I. M., Knoth, S., Montgomery, D. C., & Jones-Farmer, L. A. (2024). Introducing ChatSQC: Enhancing statistical quality control with augmented AI. Journal of Quality Technology, 56(5), 474-497. - Megahed, F. M., Chen, Y. J., Colosimo, B. M., Grasso, M. L. G., Jones-Farmer, L. A., Knoth, S., Hongyue Sun, & Zwetsloot, I. (2025). Adapting OpenAI's CLIP model for few-shot image inspection in manufacturing quality control: An expository case study with multiple application examples. arXiv preprint arXiv:2501.12596. --- class: inverse, center, middle # Structured Text Extraction for SQC Applications --- ## What is Structured Text Extraction (STE)? **Structured Text Extraction** is the process of extracting relevant information from unstructured text data and converting it into a structured format that can be easily analyzed. <br> <table class="table table-striped table-hover table-condensed" style="font-size: 12px; color: black; font-family: Arial; width: auto !important; "> <caption style="font-size: initial !important;">2025 Models Recall Data from NHTSA.</caption> <thead> <tr> <th style="text-align:right;"> Record ID </th> <th style="text-align:left;"> Defect Description </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 300365 </td> <td style="text-align:left;"> Polestar Automotive USA, Inc. (Polestar) is recalling certain 2021-2025 Polestar 2 vehicles. The rearview camera image may not display when the vehicle is placed in reverse. </td> </tr> <tr> <td style="text-align:right;"> 299665 </td> <td style="text-align:left;"> Mercedes-Benz USA, LLC (MBUSA) is recalling certain 2023-2024 AMG GT 63 S E Performance 4-door coupe, 2025 AMG GLC 63 S E Performance, GLC 63 S E Performance Coupe, AMG GT 63 S E Performance, 2023-2025 AMG S 63 E Performance, and 2024-2025 AMG SL 63 S E Performance vehicles with a hybrid powertrain. A software error may cause the high-voltage starter alternator control unit to fail, causing a loss of drive power. </td> </tr> <tr> <td style="text-align:right;"> 299911 </td> <td style="text-align:left;"> Thor Motor Company (TMC) is recalling certain 2023-2026 Axis, Chateau, Four Winds, 2023-2025 Coleman, Echelon, Freedom Elite, Geneva, Outlaw, Quantum, Vegas, 2025 Eddie Bauer, Freedom Traveler, Pasadena, and 2024-2025 Magnitude motorhomes. The slide-Out room can be deployed without the parking brake engaged, allowing the room to extend while the vehicle is in motion. </td> </tr> </tbody> </table> .footnote[ <html> <hr> </html> **Data Source:** [NHTSA Recalls Dataset From 2020-2024 (2025).](https://static.nhtsa.gov/odi/ffdd/rcl/RCL_FROM_2020_2024.zip) The data is based on the NHTSA's recall database, which contains information about vehicle recalls in the United States. **Context:** Manufacturers who determine that a product or piece of original equipment either has a safety defect, or is not in compliance with federal safety standards, are required to notify NHTSA within five business days. NHTSA requires that manufacturers file a defect and noncompliance report as well as quarterly recall status reports, in compliance with Federal Regulation 49 (the National Traffic and Motor Safety Act) Part 573, which identifies the requirements for safety recalls. NHTSA stores this information and the data can be used to search for recall information related to specific NHTSA campaigns and product types. **Note:** Recalls made in 2024 can influence later models, and hence, we are using the 2025 model year as a filter in the print out above. ] --- ## Our No-Code App for STE: A Demo Our [no-code app](https://huggingface.co/spaces/fmegahed/structured_text_extraction) utilizes the `ellmer` package to extract structured data from unstructured text, and OpenAI's `gpt-4o-mini` model to perform the extraction. <img src="figs/hf_ste.png" width="90%" style="display: block; margin: auto;" /> --- ## Some Potential Applications of STE in SQC - **Defect Detection**: Automating the identification of defects in manufacturing processes by analyzing inspection reports and maintenance logs. - **Customer Feedback Analysis**: Analyzing customer feedback, reviews, and warranty claims to identify common defects or issues with products. - **Regulatory Compliance**: Extracting information from regulatory documents, such as safety recalls or compliance reports, to ensure that products meet industry standards and regulations. - **Research:** Analyzing research papers, patents, and technical reports to extract relevant information about new materials, processes, or technologies. --- ## Practical Considerations > LLMs are **inherently stochastic**. It is important to check the: (a) **consistency** of the extracted data, and (b) **external validity** of the results. <img src="figs/intra_dist.png" width="45%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Note:** Our work on LLM consistency is available at [arXiv](https://arxiv.org/abs/2505.14918). Our detailed analysis in this area is available [here](https://fmegahed.github.io/research/llm_consistency/llm_consistency.html). ] --- class: inverse, center, middle # ChatISA: Our In-House Bot for Students <br> .pull-left-2[<br>Megahed, F. M., Chen, Y. J., Ferris, J.A., Resatar, C., Ross, K., Lee, Y., & Jones-Farmer, L. A. (2024). ChatISA: A Prompt-Engineered Chatbot for Coding, Project Management, Interview and Exam Preparation Activities. Under review. [Freely available @ [arXiv](https://arxiv.org/abs/2407.15010)].] .pull-right-2[<div><img src="figs/paper3_qr_code.png" class="paper-img" width="300px" align="right"></div>] --- ## A Live Demo of ChatISA <center> <a href="https://chatisa.fsb.miamioh.edu/"> <img alt="The interface to our ChatISA app" src="figs/chatisa_demo.gif" style="width:80%; height:80%;"> </a> </center> .footnote[ <html> <hr> </html> **Note:** We encourage the audience to experiment with **ChatISA** at <https://chatisa.fsb.miamioh.edu/>. If we have time, we can also go over [this pre-recorded and sped-up demo of our voice-enabled Interview Mentor](https://www.loom.com/share/896d4ab0e18747f0bca8dba5fff9cc36?sid=24872af4-aefd-4889-8083-b3eeb1082c52). ] --- class: inverse, center, middle # ChatSQC: Our Grounded App, to address Imprecise SQC Answers and Hallucinations <br> .pull-left-2[<br>Megahed, F. M., Chen, Y. J., Zwetsloot, I., Knoth, S., Montgomery, D.C., & Jones-Farmer, L. A. (2024). Introducing ChatSQC: Enhancing Statistical Quality Control with Augmented AI. *Journal of Quality Technology*, 56(5), 474-497. [Freely available @ [arXiv](https://arxiv.org/pdf/2308.13550)].] .pull-right-2[<div><img src="figs/paper2_qr_code.png" class="paper-img" width="300px" align="right"></div>] --- ## The Construction of ChatSQC <img src="figs/ChatSQC_flowchart_new.png" alt="The construction of ChatSQC involved four main phases: (a) a one-time extraction of the reference material, (b) a one-time preprocessing of the extracted material, (c) a continuous (online) chat inference, and (d) the hosting/deployment of the app on a web server." width="80%" style="display: block; margin: auto;" /> --- ## A Live Demo of ChatSQC <center> <a href="https://chatsqc.osc.edu/"> <img alt="The interface to our ChatSQC app" src="figs/chatsqc_demo.png" style="width:90%; height:90%;"> </a> </center> .footnote[ <html> <hr> </html> **Note:** We encourage the audience to experiment with **ChatSQC** at <https://chatsqc.osc.edu/>. ] --- class: inverse, center, middle # Adapting OpenAI's CLIP Model for Few-Shot Image Inspection in Manufacturing Quality Control <br> .pull-left-2[<br>Megahed, F. M., Chen, Y. J., Colosimo, B. M., Grasso, M. L. G., Jones-Farmer, L. A., Knoth, S., Sun, H. & Zwetsloot, I. (2025). Adapting OpenAI's CLIP Model for Few-Shot Image Inspection in Manufacturing Quality Control: An Expository Case Study with Multiple Application Examples. arXiv preprint arXiv:2501.12596. [Freely available @ [arXiv](https://arxiv.org/pdf/2501.12596)].] .pull-right-2[<div><img src="figs/paper4_qr_code.png" class="paper-img" width="300px" align="right"></div>] --- ## The Central Role of Images in SQC .center[ <video width="80%" controls> <source src="figs/image_qc_full.mp4" type="video/mp4"> </video> ] .footnote[ <html> <hr> </html> **Note:** This video was generated by Fadel M. Megahed using [Google's text-to-video Veo 3 model](https://labs.google/fx/tools/flow/). The prompts used to create and stitch the generated videos into one comprehensive video can be accessed [here](https://github.com/fmegahed/fmegahed.github.io/blob/master/talks/isspm2025/figs/image_qc_video_steps.md). ] --- ## What is CLIP? <img src="figs/CLIP.png" width="100%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Note:** CLIP (Contrastive Language–Image Pretraining) is a neural network architecture that learns to connect images and text. It was developed by OpenAI and released in January 2021. **Image Source:** <https://github.com/openai/CLIP>. ] ??? This is the main training loss for CLIP. The key idea is simple: we want the embedding of an image and its correct caption to be close, and for all other captions to be far away — and vice versa. So for each image–text pair in the mini-batch, we compute how similar the image is to every text in the batch, and how similar the text is to every image. Then we use a softmax to convert those similarities into a probability distribution, and apply cross-entropy loss to maximize the likelihood of the correct match. The loss is symmetric — we do this from image-to-text and text-to-image, and average the two directions. This simple structure is what gives CLIP its ability to generalize so broadly. --- ## Our Few Shot QC Framework <img src="figs/clip_framework.png" width="100%" style="display: block; margin: auto;" /> > **Practical Notes:** > > All preprocessing shown in the figure—center-crop → resize → patch extraction → embedding—happens **automatically** inside the standard `model, preprocess = clip.load("ViT-L/14", device=device)` function. > > In practice, you call one function, and CLIP handles the crop, resize, non-overlapping patching, and projection to the `\(d\)`-dimensional vector. > >We do **NOT** leverage the **text encoder** in our few-shot approach. We only use the **image encoder** to extract the `\(d\)`-dimensional vector representation of the image. --- ## An Example with Stochastic Textured Surfaces (STS) <img src="https://raw.githubusercontent.com/fmegahed/qe_genai/main/results/exp03_learning_images.gif" width="100%" style="display: block; margin: auto;" /> --- ## Zero-Shot Failure for the STS Example CLIP fails in **zero‑shot** because STS defects are subtle & domain‑specific. <br> | Accuracy | Sens | Spec | Prec | F1 | AUC | |---------:|-----:|-----:|-----:|---:|----:| | 0.500 | 0.000 | 1.000 | 0.000 | 0.000 | 0.171 | <br> Therefore we evaluate **few‑shot** performance while varying the learning‑set siz and two image encoder models (*ViT‑L/14* vs *ViT‑B/32*). --- ## Few-Shot Learning for the STS Example <img src="figs/exp03_comparing_two_clip_models.png" width="72%" style="display: block; margin: auto;" /> ??? - *ViT‑L/14* reaches **97% accuracy** with only **50** images/class. - Smaller *ViT‑B/32* saturates around 76%. - Fine‑grained `\(14\times14\)` patching better captures local stochastic texture. --- ## Our No-Code App for CLIP: A Demo **Our tool is available at**: <https://huggingface.co/spaces/fmegahed/clip>. <img src="figs/hf_clip.png" width="90%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Notes:** Our source code for the app can be found under our [HF Files](https://huggingface.co/spaces/fmegahed/clip/tree/main). Furthermore, the code for our detailed experiments with CLIP is available in [this Python Notebook](https://github.com/fmegahed/qe_genai/blob/main/notebook/image_inspection_with_clip.ipynb). Our under development Python library can be accessed at [FewShotIQ](https://test.pypi.org/project/FewShotIQ/0.1.1/). ] --- class: inverse, center, middle # Concluding Remarks --- ## 1. Keeping up with AI Developments is Hard!! <img src="figs/timeline_animation.gif" width="78%" style="display: block; margin: auto;" /> --- ## 2. The Fear of Technological Unemployment is Not New <img src="figs/cnn_ai_displace_jobs.png" alt="41% of companies worldwide plan to reduce workforces by 2030 due to AI" width="92%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Article Source:** Dmitracova, O. (2025, January 8). **41% of companies worldwide plan to reduce workforces by 2030 due to AI**. CNN Business. <https://www.cnn.com/2025/01/08/business/ai-job-losses-by-2030-intl/index.html> ] --- count: false ## 2. The Fear of Technological Unemployment is Not New > "The **fear** has even been expressed by some that **technological change** would in the **near future not only cause increasing unemployment**, but that **eventually it would eliminate all but a few jobs**, with the major portion of what we now call **work being performed automatically** ..." > > .font70[National Commission on Technology, Automation and Economic Progress (1966). Technology and the American Economy. Volume 1. February 1966. Report to the President and Members of Congress [(p.xii)](https://files.eric.ed.gov/fulltext/ED023803.pdf)]. -- <br> > The .black[.bold[basic fact]] is that **technology eliminates jobs, not work**. > > .font70[National Commission on Technology, Automation and Economic Progress (1966). Technology and the American Economy. Volume 1. February 1966. Report to the President and Members of Congress [(p.9)](https://files.eric.ed.gov/fulltext/ED023803.pdf)]. ??? We have the honor to present the report of the National Commission on Technology, Automation, and Economic Progress. This Commission was established by Public Law 88-444, which was approved by Congress on August 5, 1964, and signed by the President on August 19, 1964. The Commission was appointed by the President in December 1964, and the appointments were approved by the Senate on January 27, 1965. --- ## 3. The Future of Work: Projected Needed Skills <img src="figs/wef_skills_matrix.png" width="65%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Source:** World Economic Forum (2025). [The Future of Jobs Report 2025](https://reports.weforum.org/docs/WEF_Future_of_Jobs_Report_2025.pdf), P. 41. The Future of Jobs Report 2025 brings together the perspective of over 1,000 leading global employers, representing more than 14 million workers across 22 industry clusters and 55 economies from around the world. ] --- ## 4. Use Cases Overlap with our Discpline!! <img src="figs/use_cases2.svg" width="100%" style="display: block; margin: auto;" /> .footnote[ <html> <hr> </html> **Created By:** Fadel Megahed based on the text in the article by Ava McCartney. (2024). "When Not to Use Generative AI", *Gartner*. The article was published on April 23, 2024 and last accessed on October 18, 2025. It can be accessed at <https://www.gartner.com/en/articles/when-not-to-use-generative-ai>. ] --- ## Thank You! .pull-left[ - This presentation was created based on Dr. Fadel Megahed's presentation at [International 15th Workshop on Intelligent Statistical Quality Control](https://statistische-woche.de/en/startseite-en). Click [here](https://fmegahed.github.io/talks/isqc2025/genai.html) to find the original presentation. - Please do not hesitate to contact me (Tessa Chen) at <a href="mailto:ychen@udayton.edu"><i class="fa fa-paper-plane fa-fw"></i>&nbsp; ychen4@udayton.edu</a> for questions or further discussions. ] .pull-right[ <img src="./figs/Tessa_grey_G.gif" width="60%" style="display: block; margin: auto;" /> ]