This one is fairly simple and is mostly on the Lugito side. We should trigger test builds for packages when a diff is created.
We have the tooling for this already; we can simply add some code to Lugito to trigger Jenkins, and enable a Phabricator plugin on that end. Instructions are in the README here.
The non-trivial work involves creating a whole new set of builds on the Jenkins end to reflect diffs. Something like CODENAME_diff_PACKAGENAME. The complexity here comes from the different kind of branches the diff can be proposed against. That's where this can get wild. We could just do diff_PACKAGENAME and have the branch dynamically passed via the webhook (brokered by Lugito)? That'll require a bit of work.
The other bit of non-trivial work is where to build the test packages. We could just build them on Jenkins, or we could trigger LP builds. I have no preference, but however we do it, it needs to be fairly robust.
The end result of this will be that *all* diffs need to pass Harbormaster before they're merged. We can then review build logs.