Hadoop JobTracker & TaskTracker Web Interface
JobTracker (JT)
The JobTracker web interface provides a wealth of information on jobs and tasks that are running on the cluster as well as historical information on completed jobs (including ones that failed).
There are no running jobs in the screenshot above but if there were you would be able to view the current status of such jobs (including counters for input / output records read, locality of task execution etc).
At the bottom of this page is a link to the Job Tracker History viewer which allows you to view statistics on completed jobs.
After selecting a job you can drill down into the statistics for the job.
Clicking the analyse job history link on job details history page, displays a summary of task performance statistics.
Details of individual task attempts:
TaskTracker (TT)
The TaskTracker web interface provides information on the tasks that are running on the tasktracker and allows you to browse the logs for the tasktracker.
The logs directory shows the logs generated for map and reduce tasks that were run on the TaskTracker.
The URL http://<tasktracker-host>:50060/jmx displays the metrics and their values exposed via JMX for the TaskTracker.
Hi Vijay,
ReplyDeleteThanks for the article.I have a doubt, from where the Job tracker web page is getting the information about the wealth of the job.
It is from the log files ?
Please provide the solution for this query.
Hi Krishna, sorry for the very delayed response - I've not had a chance to keep my blog up to date. The Job Tracker maintains information about jobs in memory (by default it maintains up to the last 100 jobs) this is where the UI gets it's information from. It also stores historical information in the location specified by the configuration parameter hadoop.job.history.location
ReplyDelete