I'd like some opinions on the best way to tackle this. Here are the metrics I'd like to record on a regular basis:

• ci.lubuntu.me:
  • Number of jobs on ⚠️ FIX ME ⚠️ and the percentage of total jobs that is.
  • Unstable vs failing jobs and each percentage as part of ⚠️ FIX ME ⚠️
• phab.lubuntu.me:
  • Number of open tasks, with percentages of:
    • Assigned/unassigned tasks.
    • Touched in the last 30 days.
    • Urgencies.
  • Open diffs and wait times.
  • Top committers for all repositories.
• discourse.lubuntu.me:
  • Open topics in Support (they're automatically closed after 30 days and they can be closed manually after an issue is solved).
  • Top repliers and posters.
• iso.qa.ubuntu.com:
  • Number of QA tests completed, bugs attached (if any), top testers.

I originally wanted to just add this functionality to Lugito, but this seems way outside the scope of that. We're going to need a database, a Flask app, and a bunch of different modules to this program at minimum.

All of the base code is there, and I feel familiar enough adding a lot of this to Lugito to take it on myself. What I'm looking for is specific architectural feedback. SQLite vs postgresql vs etc., what kind of data to record, how often to do it, etc.

Documentation is available here. Code is available here: rMETRICS Lubuntu Project Metrics