If the power goes out while updating the operating system, after booting into recovery mode you need to check the integrity of the OS.

Plan to restore OS integrity

1. Restoring network operation
2. Cleaning up the local package repository
3. Generate packet checksums
4. Checking checksums of packages and configuration files
5. Fixing problems with packages
   5.1 Finishing configuration of packages that are installed but not configured
   5.2 Reinstalling packages that have had checksum changes
   5.3 Fixing broken packages
6. Disable third-party kernel modules
7. Final check of OS integrity using dpkg -V
8. Listing and viewing changed configuration files
9. Updating package versions
   10.Recovery of Snap and Flatpak
   11.Reboot

1. Restoring the network

You need to connect the Ethernet cable and start the network card

ifconfig -a | less
sudo nano /etc/network/interfaces

auto ensp2s0
iface enp2s0 inet dhcp
nameservers 8.8.8.8

sudo dhclient enp2s0
sudo resolvconf -u
sudo service networking restart
ping ya.ru

2. Clear package cache

sudo apt-get clean

This command clears the local repository of received package files. Cleaning is necessary to generate checksums for packages not from the local package cache, where they could have been spoofed.

3. Generate packet checksums

Installing the debsums utility

sudo apt install debsums

Initializing debsums (Ubunti Linux only).

sudo debsums_init 

4. Verifying checksums of packages and configuration files

To check all files, including configuration files:

sudo debsums --all --changed --silent

or for short

sudo debsums -acs

To check only binary files, excluding configuration files:

sudo debsums --changed --silent

or for short

sudo debsums -cs

Switch -a (or --all) to check each file, including the configuration file.
The -c switch displays only changed packages (same as --changed),
the -s switch (the same as "--silent") suppresses the output of lines where there are no changes.

sudo debsums --all --changed --silent | grep -v '.png*\|.svg*'

The second option is more preferable,without checking configuration files, excluding changes to png and svg resource files from the output:

sudo debsums -cs | grep -v '.png*\|.svg'

where is the key -v means to invert the selection - i.e. select "everything except".

Explanation: using grep in the command output, I excluded mentions of png images. and svg.
The pictures relate to the design theme, for example, in my case, to the adwaita-icon-theme-full package (how to find out the package name by the file name is described in paragraph 5.2).

5. Troubleshoot package problems

5.1 Completing package configuration

This command forces configuration of packages that have been unpacked but not configured. The -a switch tells the system to process all unpacked but not configured packages.

sudo dpkg --configure -a

5.2 Reinstalling packages that have changed their checksums

Packages are restored one by one with the command sudo apt --reinstall install package name. First you need to find the package name.

Typically, the package name is listed in the output of the debsums command on the right side in parentheses (for example, the text "from amdgpu-install package"), which means "from the amdgpu-install package".

For files where the package name is not specified, the package name can be determined by the name of the file whose checksum was changed.

For example, if the output missing file /etc/dhcp/dhclient.conf you can determine which package this file belongs to:

dpkg --search fqdn

where fqdn is the full path to the modified file, for example,/etc/dhcp/dhclient.conf

In this example, the answer is:
isc-dhcp-client

Reinstall a separate package:

sudo apt --reinstall install package_name

or

sudo apt-get --reinstall install package_name

where: package_name is the name of the package.

For example:

sudo apt --reinstall install isc-dhcp-client

If the package is no longer required (a "tail" from the previous configuration), you can uninstall it. In my case, the AMD video card is not used and I ran the command to remove the package named "amdgpu-install":

sudo apt remove -y amdgpu-install

Repeat point 5.2 until the command output sudo debsums -cs will not become empty.

Automating the system repair process

sudo apt-get install -y --reinstall $(dpkg -S $(sudo debsums -cs) | cut -d : -f 1 | sort -u)

sudo apt-get install -y --reinstall $(dpkg -S $(sudo debsums -cs) |  grep -v '.png*\|.svg' | cut -d : -f 1 | sort -u)

Where the sort -u switch eliminates duplicate package names.

(Source -askubuntu.com)

Note: missing files will have to be installed manually from packages. For example, in my case, based on the results of execution sudo debsums -cs I had to reinstall the packages youtube-dl python3-pkg-resources python3-protobuf etc. manually.

sudo debsums -cs | grep -v '.png*\|.svg' | sort -u
sudo apt-get install -y --reinstall youtube-dl python3-pip python3-debian python3-pkg-resources python3-protobuf python3-levenshtein sound-theme-freedesktop

5.3 Fixing broken packages

sudo apt install -f

where -f is the same as --fix-broken

sudo apt install --fix-broken

Description; This command will try to fix a system with broken dependencies.

If the last command produces the message "post-installation script subprocess returned error" abort. " - you need to determine the package name from the output of the apt install -f command package_name and then run:

sudo rm /var/lib/dpkg/info/package_name
sudo dpkg --configure -D 777 package_name
sudo apt-get --reinstall install package_name

6. Disabling extraneous kernel modules and removing unnecessary components

When running the command debsums -cs it found the file lsb-cprocsp-rdr-64 and the ini file /etc/opt/cprocsp/config64.ini, which are not related to the Linux operating system. These files are from the external component CryptoPRO CSP, which I removed a long time ago, but they are present in the system.

Displays a list of kernel modules that are loaded when the OS starts:

lsmod

Only the first column:

lsmod | cut -d " " -f 1 | sort | less

Checking module traces cprocsp(CryptoPro) in the folder '/etc/modprobe.d/'

grep -Rnw '/etc/modprobe.d/' -e cprocsp 

The cprocsp sample was not found among the kernel modules.

Optional components are installed in the /opt folder. The contents of which need to be controlled.

I deleted the "/opt/cprocsp" folder as unnecessary.

sudo rm -rf /opt/cprocsp

Deleting a configuration file

sudo rm -rf /etc/opt/cprocsp

7. Final OS integrity check using dpkg -V

sudo dpkg -V

where -V is the switch to check all packages on the system. The command checks integrity by comparing information from installed package files and metadata information stored in the dpkg database.

As before, I find the names of packages to reinstall, but in one command “the list of arguments is too long,” so I divided it into a number of steps.

Этаp 1. Finding errors using dpkg -V

sudo dpkg -V  > ~/files_missing.txt

View the result of the work:

cat ~/files_missing.txt

Этаp 2. Search for corresponding package files:

cat ~/files_missing.txt | grep -v '\?\?\?' | sed "s/missing//"  > ~/files_missing2.txt
cat ~/files_missing.txt | grep '\?\?\?' | sed "s/??5??????//"  > ~/files_changed.txt

Here:

• in the first file files_missing2.txt files without a mask are excluded ??5??????, that is, only missing files.

• in the second list files_changed.txt files with a mask are included ??5??????- that is, modified files.

  Part 1. defining Package Names absent files to reinstall:

    while read p; do
        echo "$p" 
        dpkg -S "$p" >> ~/reinstall.txt
    done 

Part 2. defining Package Names changed files to reinstall:

    while read p; do
        echo "$p" 
        dpkg -S "$p" >> ~/reinstall.txt
    done 

Copying package names to ~/reinstall2.txt

    while read p; do
        echo "$p" 
        echo "$p" | cut -d ":" -f 1 >> ~/reinstall2.txt
    done 

View file reinstall2.txt with a list of packages to reinstall:

cat ~/reinstall2.txt

Sorting (unique) and removing commas, into the reinstall3.txt file:

sort -u reinstall2.txt | sed 's/,//g' | sed 's/ /\n/g' | sort -u  > reinstall3.txt

Reinstalling packages according to the list from the file:

while read p; do

echo "$p"
sudo apt-get install -y --reinstall "$p"
done < ~/reinstall3.txt

8. Compiling a list of changed configuration files

Compiling a list of changed configuration files:
We discard the "missing" lines.

sudo dpkg -V  | grep -v 'missing' > ~/files_changed.txt

List of files files_changed.txt should be studied. You can additionally select only rows with column "c" (changed)

cat ~/files_changed.txt |  grep " c " | cut -d " " -f 3

We study the list and check each *.conf file from the list. If you just need to reinstall a package, the easiest way is to find the appropriate package and reinstall it.

For example:

dpkg --search /etc/ld.so.conf

libc-bin: /etc/ld.so.conf

sudo apt-get install -y --reinstall libc-bin

dpkg --search /etc/sysctl.conf

procps: /etc/sysctl.conf

sudo apt-get install -y --reinstall procps

Etc.

9. Updating package versions

Solving problems with packages that were not fully installed.

sudo dpkg --configure -a

I updated the versions as usual:

sudo apt update
sudo apt upgrade -y

Troubleshooting unmet dependencies:

sudo apt --fix-broken install
sudo apt dist-upgrade

10. Snap and Flatpak recovery

In case the store snap installed, I am not aware of special commands for checking packages. You can reinstall packages:

snap list
sudo snap remove package_name
sudo snap install package_name

Multiple packages - removal:

for package_name in package_name1, package_name2, package_name3
do
    sudo snap remove $package_name
done

Several packages - installation:

for package_name in package_name1, package_name2, package_name3
do
    sudo snap install $package_name
done

Where package_name is the name of the package.

To update package versions in the snap storage, you need to run the command:

sudo snap refresh --ignore-running

Checking application packages installed from the store flatpak:

flatpak repair

Restoring flatpak is not a quick operation, it takes about ten minutes.

11. Reboot

If the file was edited in step No. 1 of the instructions /etc/network/interfaces, you need to roll back the changes: either delete the lines for starting the network card (auto enp2s0, ...) and the DNS server (dns-nameserver ...), or comment them out:

sudo nano /etc/network/interfaces

auto enp2s0

iface enp2s0 inet dhcp

nameserver 8.8.8.8

Save changes and exit: Ctrl+O, Ctrl+X.

We reboot the operating system:

sudo shutdown -r now

Source of information: askubuntu.com
Date of last edit: 09/02/2024 - section No. 10 about snap and flatpak was added.

Related publications

Stop and disable TimeShift to save disk space

Troubleshooting L2TP/IPsec connection errors "Could not add IPsec connection" and "no acceptable traffic selectors found"

Hard disk partitioning for installing Linux Mint on an old PC with BIOS in GPT mode

Updating Linux Mint 21.3 Virginia to Linux Mint 22.3 Zena

Removing the Xanmod kernel from a Linux operating system