Page MenuHomeLubuntu

Smart package removal
Open, HighPublic

Description

I had a conversation with someone on the Release Team; we apparently don't remove enough packages post-install.

When a livefs is generated, it also generates a livecd.lubuntu.manifest-remove file. Here is an example:

For example, if btrfs filesystems aren't used, the package support shouldn't be installed.

Ubiquity goes through and removes unneeded packages if they're on that list. This is a bit tricky to implement because we need the correct logic. @wxl, could you take a look at the list and help me write some pseudocode for what we should remove here?

Event Timeline

tsimonq2 triaged this task as Unbreak Now! priority.
tsimonq2 created this object with edit policy "Release Management (Project)".
tsimonq2 lowered the priority of this task from Unbreak Now! to High.Aug 19 2018, 10:20 PM
wxl added a comment.Aug 21 2018, 10:46 AM

I don't claim to understand Ubiquity (heh) but there are two pieces of code that seem to deal with this:

  1. scripts/plugininstall.py:remove_extras()
  2. scripts/install.py:generate_blacklist()
wxl added a comment.Aug 23 2018, 11:46 AM

"Pseudocode Interpretation of install.py:generate_blacklist()" or "What It Looks Like When Your Head Explodes" by Walter Lapchynski

# note that remove is the current system, so desktop can kind of be ignored
manifest_remove = /path/to/filesystem.manifest-remove
manifest_desktop = /path/to/filesystem.manifest-desktop
manifest = /path/to/manifest

# using above files, populate desktop_packages (standard) or difference, respectively
# curiously, desktop_packages never gets used outside this section so i'm not sure why it's set
# especially for the default method
if manifest_remove
  for line in manifest_remove
    if not null or comment
      difference += first column of (first column of line before colon) before space
  for line in manifest
    if not null or comment
      live_packages += first column of (first column of line before colon) before space
  desktop_packages = live_packages - difference
else if manifest_remove
  for line in manifest_desktop
    if not null or comment
      desktop_packages += first column of (first column of line before colon) before space
  for line in manifest
    if not null or comment
      live_packages += first column of (first column of line before colon) before space
  difference = live_packages - desktop_packages
else
  difference = emptyset

# add to difference files from minimal install's remove list
if minimal_install
  for line in /path/to/manifest-minimal-remove
    difference += first column of line before colon

# add (but don't duplicate) to difference anything installed and in restricted
if apt-setup/restricted
  for package in cache
    if package is installed and in restricted
      difference |= package

# add to keep anything in /var/lib/ubiquity/apt-installed
keep = query_recorded_installed()

# populate keep with necessary files for bootloading
arch, subarch = archdetect()
if arch is amd64 or i386
  keep += 'grub-pc'
  if subarch is efi
    keep += 'grub-efi', 'grub-efi-amd64', 'grub-efi-amd64-signed', 'shim-signed', 'mokutil'
    altmeta base-installer/kernel/altmeta
    if altmeta
      altmeta = '-$altmeta'
    keep += 'linux-signed-generic$altmeta'
  else
    keep += 'grub'
else if arch is armel or armhf and subarch is omap, omap4, or mx5
  keep += 'flash-kernel', 'u-boot-tools'
else if arch is powerpc
  keep += 'yaboot', 'hfsutils'

# add ubiquity for oem-config
if oem-config
  keep += ubiquity

# subtract from difference packages that can't be removed if we want to keep those in keep
difference -= expand_dependencies_simple(cache, keep, difference)

# keep only those packages in difference where there's no prerm
difference = 
  for package in difference
    if /var/lib/dpkg/info/$package.prerm does not exist

# make difference equal to those packages that aren't going to mess everything up, essentially
# this code and the get_remove_list function is enough to drive a person insane
with cache
  for package in difference
    if package in confirmed_remove
      continue
    would_remove = get_remove_list(cache, pkg)
    if would_remove <= difference
      confirmed_remove |= would_remove
    else
      for package in would_remove
        mark package for keeping
difference = confirmed_remove

# if there are no more packages left to remove, quit
if difference is empty
  delete cache
  empty blacklist
  end

# generate a blacklist of files for each package in difference
cmd = `dpkg -L $difference`
for file in cmd
  u[file] = 1
blacklist = u
tsimonq2 removed tsimonq2 as the assignee of this task.Sep 19 2018, 10:12 AM
tsimonq2 edited projects, added Restricted Project; removed Release Management.Thu, Oct 4, 9:32 AM