tldr - powered by Generative AI

• Apache OpenNLP is a Java-based NLP tool that has been around for over a decade and offers various capabilities such as tokenization, document classification, and named entity recognition
• Apache Solr depends on Apache Lucene for search functionality, and Apache Lucene has a dependency on Apache OpenNLP for some NLP operations
• The ONNX Runtime allows for the use of deep learning models across programming languages, architectures, and platforms, enabling the use of NLP services created in other languages
• The speaker demonstrates how a deep learning model trained using PyTorch or Tensorflow can be used for inference from a Java search stack of Apache OpenNLP, Apache Lucene, and Apache Solr
• The speaker discusses the challenges and relationships between OpenNLP, Lucene, and Solr, and provides resources for attendees to get started with these open source projects

tldr - powered by Generative AI

• Running ML server locally is important to ensure everything works and debug any issues before deployment to production.
• Other resources for CI/CD for production machine learning at scale, production machine learning monitoring, machine learning security, and machine learning ecosystem and operations.
• Collaboration with Hugging Face team to access a pre-trained GPT2 model using their Transformers library.
• Optimization of the model using ONNX serialization format.
• Deployment to Kubernetes cluster after testing locally to ensure it works.
• Anecdote about a computationally intensive dungeon crawler game that uses AI model for personalization.