Navigating the Future of Observability: A Deep Dive with Alok Gupta

Alok Gupta’s journey into the field of observability and Site Reliability Engineering (SRE) is a story of continuous growth and innovation. Starting at Persistent Systems in Pune, India, Alok spent over a decade honing his skills in automation, particularly in functional test cases. "I focused on automating large-scale test scenarios, identifying performance bottlenecks, and improving test coverage," he recalls. His early experience laid a strong foundation for the specialized roles he later embraced, particularly in observability and automation.

Upon moving to the United States, Alok's expertise evolved further. At Aeris Communication, he developed a machine learning-based time series forecasting system, which minimized manual dependencies and infrastructure outages. “The system predicted failures before they could escalate, enabling us to take proactive measures,” he explains. This shift to predictive monitoring marked a key milestone in his career. Later, at Box Inc., he designed high-volume distributed logging pipelines, managing data from diverse sources. "Handling 450 TB of data daily from sources like Kubernetes and GCE posed challenges, but with OTEL agents and Edge Delta, we reduced ingestion by 90%, which lowered both MTTD and MTTR," Alok shares.

Alok also led the challenging migration from On-Prem Splunk to Cloud Splunk. "The key was meticulous planning to ensure data continuity and system performance during the transition. Optimizing configurations and integrating new cloud functionalities significantly improved our observability capabilities," he reflects on this complex project.

His experience extends to the implementation of Auto Healing on alerts at Aeris Communication, which automated infrastructure recovery. “The Auto Healing mechanism reduced downtime and human intervention, making our systems more resilient,” Alok explains. Furthermore, his work on reducing ELK incidents by 90% through performance optimizations and lifecycle management was another game-changer. “We focused on optimizing storage with Index Lifecycle Policies and fine-tuning the ELK stack for better reliability,” he adds.

Developing Prometheus-based Grafana dashboards at Aeris Communication was another key aspect of his role. "The goal was to create actionable, real-time insights for teams to monitor microservices," he says. By integrating metrics from multiple sources and ensuring the dashboards were intuitive, Alok helped teams quickly identify and resolve issues.

Alok's experience with test automation at Synchronoss Technologies, particularly for IMAP, POP, and SMTP, was another notable achievement. “I automated RFC compliance tests using Python, reducing manual testing and improving test coverage,” he explains. This automation was pivotal in delivering high-quality software with fewer defects. At Persistent Systems, Alok reached an impressive 80% test code coverage by automating over 3,000 functional test cases. "Automation allowed us to uncover defects early, ensuring stable and reliable software," he says.

Mentoring junior engineers at Box Inc. is something Alok takes seriously. “I emphasize the core principles of observability, like metrics, logs, and traces. Hands-on training with tools like ELK, Prometheus, and Grafana is key,” he explains. By encouraging ownership of projects and providing continuous feedback, Alok ensures that the next generation of engineers is well-equipped to excel in the field.

Alok Gupta’s career is a testament to the power of innovation, adaptability, and collaboration in the ever-evolving tech landscape. His contributions to observability and SRE continue to set high standards for reliability and efficiency in the field.