[clug] [anne.a.gunn at gmail.com: Ubuntu problem] Dist-upgrade suggestions

Scott Ferguson scott.ferguson.clug at gmail.com
Fri Jul 31 17:13:04 UTC 2015

On 31/07/15 19:25, John Mills wrote:
> Ubuntu Desktop 14.04
> AARNet mirror was down last Sunday afternoon.
> I switched to Internode mirror and was able to get a good update.
> I don't know how automatic updates would have handled this, as I do mine manually.
> There was also what appeared to be an error in the apt mechanisms on my machine.
> On 24 July a panel indicator appeared with text:
> An error occurred....could not open file 
> /var/lib/apt/lists/au.archive.ubuntu.com_ubuntu_dists_trusty-updates_restricted_binary-amd64_Packages - open (2: No such file or directory...
> I couldn't find this in the logs - I only have a screenshot of it.
> I ran apt-get update (after I had a good repository) and I think that fixed it.
> Several kernels have come down in the last few days.
> The most recent update using the Software Updater GUI required authentication to boot.
> "It's never done that before".

It's too late to be useful to you now. But may prove useful to others,
and you next time. Steps 11 & 12 will help you avoid one of the issues
you encountered.


I haven't had anyone else proof-read the following - exercise due care.

=====Basic Steps to a Problem-Free Dist-Upgrade=====

To reduce problems during a dist-upgrade with a distro that uses apt
package management I recommend doing the following as a minimum.

Step through all of the Footnotes too. Read the Footnote for each Step
*before* doing the Step:-

*1.* Read the upgrade notes carefully for your distro. Not all
dist-upgrades are equal.

*2.* Backup ~
If there's a lot just do your .files and .directories (but not linked
files or caches)

*3.* Backup all of /etc/apt/sources.list.d/* and then clear it. You can
copy them back after a successful dist-upgrade *and* change the release
names as appropriate before re-installation.
Double check /etc/apt/sources.list and move any non-standard repos to
separate /etc/apt/sources.list.d/*.list fragment files to simplify repo
management [*1]

*4.* Backup /etc

*5.* Remove any packages from non-standard repositories they will likely
cause problems. Also remove any packages from other releases - they
*will* cause upgrade problems[*2]

*6.* Check that you don't have pinning that will cause a problem [*3]

*7.* Remove any orphaned packages - it 'may' reduce the download, and
will save space and reduce risks of dist-upgrade problems [*4]

*8.* Double-check which repositories you have enabled, and remove
non-standard repositories [*5]

*9.* Update your current version. It's not absolutely necessary, but
it's the safest approach as the dist-upgrade of up-to-date packages has
had more testing [*6]

*10.* Edit /etc/apt/sources.list and change the version *name* to the
latest version [*7]

*11.* Check that you have enough free space for the upgrade [*8]

*12* *Simulate* a download of all the packages needed for the upgrade.
This step will show you how much will be downloaded. Check the total
amount against your figure from Note 8. [*9]

*13.* If the you had no errors, and sufficient space, proceed with the
dist-upgrade using the package manager of your choice. I recommend using
apt instead of a gui package manager so that you can record the process
with script. The upgrade will use the packages already downloaded - so
if the network drops out for any reason the upgrade can continue without
a network[*10]

Footnotes are attached to this email so that the code in it is easy to
copy and paste.


Suggestion: someone might like to do a version of the above process for
a non-apt-management system

I hope that helps someone.

> Hope this helps.
> John Mills
> On Fri, Jul 31, 2015 at 11:29:59AM +1000, Tony Breeds wrote:
>> Date: Fri, 31 Jul 2015 11:29:59 +1000
>> From: Tony Breeds <tony at bakeyournoodle.com>
>> Subject: [clug] [anne.a.gunn at gmail.com: Ubuntu problem]
>> To: linux at lists.samba.org
>> Cc: A A Gunn <anne.a.gunn at gmail.com>
>> Mail-Followup-To: linux at lists.samba.org, A A Gunn <anne.a.gunn at gmail.com>
>> Hi All,
>>     Is anyone able to help with this?
>> ----- Forwarded message from A A Gunn <anne.a.gunn at gmail.com> -----
>> Date: Fri, 31 Jul 2015 11:02:22 +1000
>> From: A A Gunn <anne.a.gunn at gmail.com>
>> To: tony at bakeyournoodle.com
>> Subject: Ubuntu problem
>> Dear Tony,
>> Found the clug online with this tablet after my pc stopped during a
>> download of 14.04 ubuntu.

Apropos of which - this is probably a relevant time for a rant ;) as
part of my business is Linux support.
That's *not* a plug for business, home computers are not my sector,
though I occasionally, selectively, will provide support for problems I
see on this list (so please don't contact me directly, any thing to this
email address not from the list is filtered directly to the big round
grey filing cabinet - I read the list, if I have time I'll contact you).
I prefer pro bono - except when someone has had a "bit of a fiddle", and
failed. Hence the rant.

[rant specifically *not* aimed at Chris]
The road to hell is paved with good intentions. Please: don't change
system files without making a backup, or fail to take notes on what you
tried - it just it hard for whoever has to clean up the mess, and the
poor user whose box you trashed. It certainly won't help your employment
If you 'tried' 10 things to fix 1 problem - and failed, there are now 11
things to hunt down and check, and likely as many problems.
And no, (you know who you are) you don't install system software into
the home directory, you especially don't change the permissions to root,
and you should never do that when the same software has been installed
to the correct location by the package manager. A spaghetti of symlinks
does not make it OK. Nor does the order of processing $PATH negate
long-term management problems.
Pro Tip - there are no eyes in your fingers. It's not Schrödinger's cat
- having a look shouldn't change anything.
If you want help doing something you don't know how to do, don't insist
on telling people how to (fix what you don't how to do).

Kind regards

-------------- next part --------------
Unless otherwise noted run the commands as a general user

[*1] Keep notes of your dist-upgrade process - even if it's a failure free the notes may be useful.
You will need lsb and script installed.
Check if they are installed:-
dpkg --get-selections|grep 'lsb\|util-linux'

If either are not installed, install them. Choose the command/s to run from the following lines. As root:-
apt-get -y install lsb
apt-get -y install util-linux

Create a directory to neatly store your dist-upgrade notes:-
mkdir ~/Dist-Upgrade

Record a list of your currently used repositories:-
grep '^[^#]' /etc/apt/sources.list{,.d/*} > ~/Dist-Upgrade/old_repositories

Pro Tip: put your notes about installing the signing key for that repository into comments in the *.list fragment file
cat /etc/apt/sources.list.d/qgis.list
#gpg --recv-key DD45F6C3
#gpg --export --armor DD45F6C3 | apt-key add -
deb http://qgis.org/debian wheezy main
# deb-src http://qgis.org/debian wheezy main

Move all your non-standard repos lists to backup. As root:-
mv /etc/apt/sources.list.d/* ~/Dist-Upgrade

[*2] There is no simple one-guide-for-all-variations, but the first step is to record what packages and versions you have installed from other non-standard and different release repositories so you can reinstall them after the dist-upgrade if you wish.

Method 1. Requirements: lsb must be installed (it usually is), and apt-show-versions
Note: not all packages conform to Debian packaging guidelines and show the origin tag in the package Release file. YMWV

List all installed packages that are not from the standard repositories for your current release, or backports:-
apt-show-versions | grep -v `lsb_release -c | cut -f2` | more

Method 2. Requirements: aptitude must be installed:-
aptitude search '~S ~i !~ODebian !~o' | more

Method 3. Use Synaptic and filters

===Removing those packages===

Carefully check the projected outcomes for removing each package - if the projections look sane, remove the package. If you foresee *any* problems post a query to the user-list for the appropriate distro with the relevant information and a useful post title.

[*3] Note: it's highly recommended that pinning be put into fragment *.pref files beneath /etc/apt/preferences.d instead of in a monolithic /etc/apt/preferences
cat /etc/apt/preferences.d/multimedia.pref
Package: *
Pin: origin *.deb-multimedia.org
Pin-Priority: 200

Move your current pinning unless you have a compelling reason not to. As root:-
mv /etc/apt/preferences{.conf,.d/*} ~/Dist-Upgrade

[*4] Requirements:- deborphan must be installed

See what packages are orphaned:-

See what packages will be removed if you uninstall orphaned packages:-
apt-get -s remove `deborphan` | more

If the projected outcome looks sane proceed with the removal. As root:-
apt-get --purge remove `deborphan`

Make a record of what packages you have now have installed:-
dpkg --get-selections > ~/Dist-Upgrade/old_selections

[*5] grep '^[^#]' /etc/apt/sources.list{,.d/*}
If you find any repositories listed in /etc/apt/sources.list that are not your currently installed release - edit them out to save possible confusion later (you'll need to be root to do that)

Requirements: lsb must be installed. To see what your currently installed release is:-
lsb_release -c

Note: always use the release *name* in repository lines, this will prevent accidentally mixing releases and breaking your system

[*6] As root:-
apt-get update && apt-get -d upgrade && apt-get upgrade

Make a record of all your currently installed packages *and* the versions in use (this will take a while - it's less than optimal bash):-
for i in `dpkg --get-selections | sed 's/\s\+/ /g' | cut -d ' ' -f 1`;do dpkg -l $i | tail -n1 | sed 's/\s\+/ /g' | cut -d ' ' -f 2;dpkg -l $i | tail -n1 | sed 's/\s\+/ /g' | cut -d ' ' -f 3;done > ~/Dist-Upgrade/all_installed_packages_and_versions

[*7] Note: replace $CurrentRelease with the *name* (not version number) of your current release (see [*5], replace "$UpradeTarget" with the *name* of the *next* release after your current release. e.g. jessie is you are currently running wheezy. It is possible to skip a release - but it's safer to take smaller steps. As root:-
sed -i 's/$CurrentRelease/$UpgradeTarget/g' /etc/apt/sources.list

[*8] Clear the package cache. As root:-
apt-get clean

Then find out how much free space you have available for downloaded packages:-
df -h /var/cache/apt/archives|tail -n1|awk '{print $2}'

[*9] Important note: at the prompt type "n" and press Enter, you don't want to do the dist-upgrade download yet - you just want to see how big the download is:-
apt-get -d dist-upgrade

[*10] If you have sufficient space, check that the projected outcome of dist-upgrade looks sane:-
apt-get -s dist-upgrade | more

If the output looks sane proceed to the next line.
Requirements: script must be installed. Replace $YourName with your regular username. As root:-
script /home/$YourName/Dist-Upgrade/dist-upgrade-record;apt-get -y dist-upgrade

When you've finished the dist-upgrade type "exit" and press Enter to end the script recording, then (while still root) changes the permissions on your notes. Replace $YourName with your regular user name. As root:-
chown -R $YourName:$YourName /home/$YourName/Dist-Upgrade

More information about the linux mailing list