Riaan's SysAdmin Blog

My tips, howtos, gotchas, snippets and stuff. Use at your own risk!

Solaris

Solaris Boot Environment Size

If you have wondered why your root file system (RPOOL) on Solaris is out of space and you have double checked and none of the usual culprits are eating your space, you may need to look if perhaps you captured something large in your snapshots. For the most part I am not taking snapshots of the root file system but if you do Solaris updates (SRU) you are definitely using snapshots. This is what I had to do to reclaim space.

Warning you are on your own if you ruin your boot OS. I have the luxury of the root OS being on a SAN LUN that have separate snapshot technology and can recover quick in case something went wrong.

Space Before

# df -h
Filesystem             Size   Used  Available Capacity  Mounted on
rpool/ROOT/solaris-3    24G   9.9G       3.9M   100%    /

Lets create a new environment

# beadm create solaris-4
# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 NR     /          20.76G static 2014-11-21 17:42
solaris-4 -      -          74.0K  static 2015-03-11 12:57
# beadm activate solaris-4
# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 N      /          67.0K  static 2014-11-21 17:42
solaris-4 R      -          20.76G static 2015-03-11 12:57

As you can see that did nothing for us. It carried the 20G over.

# zfs list -t snapshot
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-4@install                   941M      -  2.03G  -
rpool/ROOT/solaris-4@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-4@2015-03-11-19:57:34       170K      -  2.31G  -
rpool/ROOT/solaris-4/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-4/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-4/var@2015-03-11-19:57:34    77K      -   146M  -

This piece was redundant but since I was not sure if maybe I left a solaris-4 BE plus snapshot on this system from experiments before I wanted one I know for sure is brand new.

# beadm create solaris-5
# zfs list -t snapshot
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-3@2015-03-11-20:02:33          0      -  2.31G  -
rpool/ROOT/solaris-3/var@2015-03-11-20:02:33      0      -   146M  -
rpool/ROOT/solaris-4@install                   941M      -  2.03G  -
rpool/ROOT/solaris-4@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-4@2015-03-11-19:57:34       170K      -  2.31G  -
rpool/ROOT/solaris-4/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-4/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-4/var@2015-03-11-19:57:34    77K      -   146M  -
# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 N      /          790.0K static 2014-11-21 17:42
solaris-4 R      -          20.76G static 2015-03-11 12:57
solaris-5 -      -          75.0K  static 2015-03-11 13:02
# beadm activate solaris-5
# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 N      /          67.0K  static 2014-11-21 17:42
solaris-4 -      -          260.0K static 2015-03-11 12:57
solaris-5 R      -          20.76G static 2015-03-11 13:02
# zfs list -t snapshot
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-5@install                   941M      -  2.03G  -
rpool/ROOT/solaris-5@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-5@2015-03-11-19:57:34       133K      -  2.31G  -
rpool/ROOT/solaris-5@2015-03-11-20:02:33        64K      -  2.31G  -
rpool/ROOT/solaris-5/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-5/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-5/var@2015-03-11-19:57:34   211K      -   146M  -
rpool/ROOT/solaris-5/var@2015-03-11-20:02:33    48K      -   146M  -

reboot

# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 -      -          8.46M  static 2014-11-21 17:42
solaris-4 -      -          260.0K static 2015-03-11 12:57
solaris-5 NR     /          20.79G static 2015-03-11 13:02

# beadm destroy solaris-4
Are you sure you want to destroy solaris-4?  This action cannot be undone(y/[n]): y
# zfs list -t snapshot
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-5@install                   941M      -  2.03G  -
rpool/ROOT/solaris-5@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-5@2015-03-11-20:02:33      29.8M      -  2.31G  -
rpool/ROOT/solaris-5/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-5/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-5/var@2015-03-11-20:02:33  2.15M      -   146M  -

# beadm list -a solaris-5
BE/Dataset/Snapshot                             Active Mountpoint Space   Policy Created
-------------------                             ------ ---------- -----   ------ -------
solaris-5
   rpool/ROOT/solaris-5                         NR     /          11.51G  static 2015-03-11 13:02
   rpool/ROOT/solaris-5/var                     -      /var       349.62M static 2015-03-11 13:02
   rpool/ROOT/solaris-5/var@2014-06-17-16:56:58 -      -          110.55M static 2014-06-17 09:56
   rpool/ROOT/solaris-5/var@2015-03-11-20:02:33 -      -          2.15M   static 2015-03-11 13:02
   rpool/ROOT/solaris-5/var@install             -      -          90.82M  static 2013-07-09 10:30
   rpool/ROOT/solaris-5@2014-06-17-16:56:58     -      -          7.82G   static 2014-06-17 09:56
   rpool/ROOT/solaris-5@2015-03-11-20:02:33     -      -          29.77M  static 2015-03-11 13:02
   rpool/ROOT/solaris-5@install                 -      -          941.40M static 2013-07-09 10:30

# beadm list -s solaris-5
BE/Snapshot                      Space   Policy Created
-----------                      -----   ------ -------
solaris-5
   solaris-5@2014-06-17-16:56:58 7.82G   static 2014-06-17 09:56
   solaris-5@2015-03-11-20:02:33 29.77M  static 2015-03-11 13:02
   solaris-5@install             941.40M static 2013-07-09 10:30

reboot

# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris-1 -      -          8.48M  static 2014-01-16 16:52
solaris-3 -      -          8.46M  static 2014-11-21 17:42
solaris-5 NR     /          20.86G static 2015-03-11 13:02

# zfs list -t snapshot
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-5@install                   941M      -  2.03G  -
rpool/ROOT/solaris-5@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-5@2015-03-11-20:02:33      48.0M      -  2.31G  -
rpool/ROOT/solaris-5/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-5/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-5/var@2015-03-11-20:02:33  5.00M      -   146M  -

# beadm list -s solaris-5
BE/Snapshot                      Space   Policy Created
-----------                      -----   ------ -------
solaris-5
   solaris-5@2014-06-17-16:56:58 7.82G   static 2014-06-17 09:56
   solaris-5@2015-03-11-20:02:33 47.98M  static 2015-03-11 13:02
   solaris-5@install             941.40M static 2013-07-09 10:30

Ok the previous part was completely redundant but I am leaving it in since it might be valuable in general. Lets now create a clean BE from a fresh snapshot. This may also be redundant but it is worthwhile to go through this.

# beadm create solaris-5@now
# beadm list -s solaris-5
BE/Snapshot                      Space   Policy Created
-----------                      -----   ------ -------
solaris-5
   solaris-5@2014-06-17-16:56:58 7.82G   static 2014-06-17 09:56
   solaris-5@2015-03-11-20:02:33 47.98M  static 2015-03-11 13:02
   solaris-5@install             941.40M static 2013-07-09 10:30
   solaris-5@now                 0       static 2015-03-11 13:22

# beadm create -e solaris-5@now solaris-clean
# beadm list
BE            Active Mountpoint Space  Policy Created
--            ------ ---------- -----  ------ -------
solaris-1     -      -          8.48M  static 2014-01-16 16:52
solaris-3     -      -          8.46M  static 2014-11-21 17:42
solaris-5     NR     /          20.86G static 2015-03-11 13:02
solaris-clean -      -          71.0K  static 2015-03-11 13:23

# beadm activate solaris-clean
# beadm list
BE            Active Mountpoint Space  Policy Created
--            ------ ---------- -----  ------ -------
solaris-1     -      -          8.48M  static 2014-01-16 16:52
solaris-3     -      -          8.46M  static 2014-11-21 17:42
solaris-5     N      /          123.0K static 2015-03-11 13:02
solaris-clean R      -          20.86G static 2015-03-11 13:23

# zfs list -t snapshot
NAME                                               USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-clean@install                   941M      -  2.03G  -
rpool/ROOT/solaris-clean@2014-06-17-16:56:58      7.82G      -  9.31G  -
rpool/ROOT/solaris-clean@2015-03-11-20:02:33      48.0M      -  2.31G  -
rpool/ROOT/solaris-clean@now                       166K      -  2.31G  -
rpool/ROOT/solaris-clean/var@install              90.8M      -  96.7M  -
rpool/ROOT/solaris-clean/var@2014-06-17-16:56:58   111M      -   139M  -
rpool/ROOT/solaris-clean/var@2015-03-11-20:02:33  5.00M      -   146M  -
rpool/ROOT/solaris-clean/var@now                    75K      -   145M  -

And this is the heart of what needs to happen. As I said you were warned. You better know what you are doing as well as have a fallback. This is your boot volume.  Lets just get rid of that old snapshot from 2014.

# zfs destroy rpool/ROOT/solaris-clean@2014-06-17-16:56:58
cannot destroy 'rpool/ROOT/solaris-clean@2014-06-17-16:56:58': snapshot has dependent clones
use '-R' to destroy the following datasets:
rpool/ROOT/solaris-1/var
rpool/ROOT/solaris-1

# zfs destroy -R rpool/ROOT/solaris-clean@2014-06-17-16:56:58
# zfs list -t snapshot
NAME                                               USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-clean@install                  1.01G      -  2.03G  -
rpool/ROOT/solaris-clean@2015-03-11-20:02:33      48.0M      -  2.31G  -
rpool/ROOT/solaris-clean@now                       166K      -  2.31G  -
rpool/ROOT/solaris-clean/var@install              91.2M      -  96.7M  -
rpool/ROOT/solaris-clean/var@2015-03-11-20:02:33  5.00M      -   146M  -
rpool/ROOT/solaris-clean/var@now                    75K      -   145M  -
# beadm list
BE            Active Mountpoint Space  Policy Created
--            ------ ---------- -----  ------ -------
solaris-3     -      -          8.46M  static 2014-11-21 17:42
solaris-5     N      /          123.0K static 2015-03-11 13:02
solaris-clean R      -          4.99G  static 2015-03-11 13:23

Since we are cleaning up lets just get rid of this old BE also.

# beadm destroy solaris-3
Are you sure you want to destroy solaris-3?  This action cannot be undone(y/[n]): y
# beadm list
BE            Active Mountpoint Space  Policy Created
--            ------ ---------- -----  ------ -------
solaris-5     N      /          123.0K static 2015-03-11 13:02
solaris-clean R      -          4.89G  static 2015-03-11 13:23

reboot

# df -h
Filesystem             Size   Used  Available Capacity  Mounted on
rpool/ROOT/solaris-clean
                        24G   2.3G       8.5G    22%    /
[..]

<strong>Finally things look much better.</strong>

# beadm list
BE            Active Mountpoint Space Policy Created
--            ------ ---------- ----- ------ -------
solaris-5     -      -          8.25M static 2015-03-11 13:02
solaris-clean NR     /          4.95G static 2015-03-11 13:23
# beadm list -s
BE/Snapshot              Space  Policy Created
-----------              -----  ------ -------
solaris-5
solaris-clean
   solaris-clean@install 1.01G  static 2013-07-09 10:30
   solaris-clean@now     29.76M static 2015-03-11 13:22
# beadm list -a
BE/Dataset/Snapshot                     Active Mountpoint Space   Policy Created
-------------------                     ------ ---------- -----   ------ -------
solaris-5
   rpool/ROOT/solaris-5                 -      -          5.84M   static 2015-03-11 13:02
   rpool/ROOT/solaris-5/var             -      -          2.40M   static 2015-03-11 13:02
solaris-clean
   rpool/ROOT/solaris-clean             NR     /          3.59G   static 2015-03-11 13:23
   rpool/ROOT/solaris-clean/var         -      /var       238.33M static 2015-03-11 13:23
   rpool/ROOT/solaris-clean/var@install -      -          91.21M  static 2013-07-09 10:30
   rpool/ROOT/solaris-clean/var@now     -      -          2.21M   static 2015-03-11 13:22
   rpool/ROOT/solaris-clean@install     -      -          1.01G   static 2013-07-09 10:30
   rpool/ROOT/solaris-clean@now         -      -          29.76M  static 2015-03-11 13:22

# zfs list -t snapshot
NAME                                   USED  AVAIL  REFER  MOUNTPOINT
rpool/ROOT/solaris-clean@install      1.01G      -  2.03G  -
rpool/ROOT/solaris-clean@now          29.8M      -  2.31G  -
rpool/ROOT/solaris-clean/var@install  91.2M      -  96.7M  -
rpool/ROOT/solaris-clean/var@now      2.21M      -   145M  -

admin

Bio Info for Riaan