Maintaining the Athena framework at peak efficiency in terms of resource utilization is crucial. The SPOT team regularly checks Athena’s performance and steps in to investigate if there’s any performance degradation. It’s also helpful for users to be able to analyze performance themselves. This guide introduces the tools commonly used with Athena to track and understand how resources are being used.
Here you can find information about several tools used for monitoring and profiling of Athena software withing SPOT duty. All those tools collect the resource metrics such as CPU time, wall time, memory allocation, etc. We separate them here according to usage purpose: monitoring and profiling. Monitoring focuses on real-time observation of job executions and resource utilization. They offer less intrusive monitoring capabilities, enabling them to operate in real-time without significantly impacting performance. However, while they provide immediate insights into job executions and resource usage, they generally offer less detailed information compared to profiling tools.
Profiling, on the other hand, delves deeply into the execution of Athena jobs, conducting a meticulous examination to identify potential bottlenecks and optimize performance. While monitoring typically offers only a glimpse of potential issues like memory errors or leaks, profiling provides precise insights into the exact location of such problems. Nonetheless, it often requires a longer duration for analysis.