Linux LVM woes
- Thread starter guddler
- Start date
Go out and celebrate!! I'm just sending some logs and stuff off to Pobster and he's going to have a quick nose (presumably on his way home, or at home later).
Essentially though, I added a new drive earlier (this is all virtual), added it to the volume group, extended the logical volume, resized the ext partition. Phew! All good. Then I noticed that the first disk in the LV was only using 15G of it's 25G size so I reversed what I'd just done and expanded that drive to take up the full 25G. Or so I thought. I go to bring the LV back online (lvchange -ay) and it won't. It failes with an ioctl error. Looking in the syslog shows it's complaining that the drive isn't big enough for what it's being asked to do. WTF???
Since it's all virtual, I'm at the point of just wanting the data off any which way I can at the moment. This is work so I'm going to be in such a world of pain if I've lost the /data folder from the dev server
. And no, before you ask, it's not backed up. Backup? We don't need no stinking backup! That would involve money!!!
Ho Hum...
[Edit} Hmm, wonder if google can see this?guddler2011-07-12 19:04:22
Essentially though, I added a new drive earlier (this is all virtual), added it to the volume group, extended the logical volume, resized the ext partition. Phew! All good. Then I noticed that the first disk in the LV was only using 15G of it's 25G size so I reversed what I'd just done and expanded that drive to take up the full 25G. Or so I thought. I go to bring the LV back online (lvchange -ay) and it won't. It failes with an ioctl error. Looking in the syslog shows it's complaining that the drive isn't big enough for what it's being asked to do. WTF???
Since it's all virtual, I'm at the point of just wanting the data off any which way I can at the moment. This is work so I'm going to be in such a world of pain if I've lost the /data folder from the dev server
Ho Hum...
[Edit} Hmm, wonder if google can see this?guddler2011-07-12 19:04:22
That's beyond my LVM experience so I'm just watching now 
Hope you get it sorted.
Regarding backups, I invented a new system a while ago. You just run everything through SHA1 and write down the hash value onto a post-it note.
Then if the data is ever lost, I'll be stressing so badly that I'm pretty confident I'll find some way to break information theory and work back from the hash output to the input set
Google can't see this section so no worries about any work posts.
Hope you get it sorted.Regarding backups, I invented a new system a while ago. You just run everything through SHA1 and write down the hash value onto a post-it note.
Then if the data is ever lost, I'll be stressing so badly that I'm pretty confident I'll find some way to break information theory and work back from the hash output to the input set
Google can't see this section so no worries about any work posts.
Still struggling like buggery at the moment, though I know what I've done! what annoys me the most is that the tools let me. Didn't they know I'm an idiot??
Basically, the underlying disk is 25GB in size. This has a single partition of 16.1G on it (god knows why I did that!!), of type LVM. What the LVM tools have let me do is resize the PV to 20GB even though I forgot about the underlying 16.1GB partition size, increase the LV to at least 20G, if not 25G (thinking there's the new drive too). All good and recoverable so far, until that is, I extended the ext4 partition on the LV to take account of the new size. Again, it let me. Which I find particularly odd!
I now seem to be in the position where either LVM is happy but I can't mount the disk due to the syslog message that it's not big enough for the file system, or, I shrink everything in the LVM and then I can't bring it online. It's a weird one! (That may be the wrong way round - I'm tired!)
I'm just going to see if there's some way to force the mount and copy the data off onto a standard EXT4 partition. Thankfully, just after my post above (6pm) I thought to take a copy of the underlying VM so I now have the benefit of being able to experiment and revert. Wish I'd thought to do that before I started completely!!
Was at it 'till 3am so going to have a break for a min.
Basically, the underlying disk is 25GB in size. This has a single partition of 16.1G on it (god knows why I did that!!), of type LVM. What the LVM tools have let me do is resize the PV to 20GB even though I forgot about the underlying 16.1GB partition size, increase the LV to at least 20G, if not 25G (thinking there's the new drive too). All good and recoverable so far, until that is, I extended the ext4 partition on the LV to take account of the new size. Again, it let me. Which I find particularly odd!
I now seem to be in the position where either LVM is happy but I can't mount the disk due to the syslog message that it's not big enough for the file system, or, I shrink everything in the LVM and then I can't bring it online. It's a weird one! (That may be the wrong way round - I'm tired!)
I'm just going to see if there's some way to force the mount and copy the data off onto a standard EXT4 partition. Thankfully, just after my post above (6pm) I thought to take a copy of the underlying VM so I now have the benefit of being able to experiment and revert. Wish I'd thought to do that before I started completely!!
Was at it 'till 3am so going to have a break for a min.
Phew - got there! (Though in fairness, I always knew the data should have been ok!)
I think I got there by removing the addition disk from the volume group, shrinking the LV down to the extents allowed by the original disk (16G), re-adding the new disk to the VG, then finally growing the LV back to a size big enough to cover the etx4 file system (21G). Brought it back online and bingo, it mounted with all my data still there.
I'm now going to take a fresh copy of both VMDKs as backup. Then create a new partition on the first disk to allow me to use the remaining space. Add that new partition as a new PV, add the PV to the VG, expand the VG to use the full 50G and finally resize the ext4 system.
Then, if all goes well, implement a backup strategy to the production SAN so I don't have to go through this sh*t again if I screw up again.
And I thought LVM was meant to make life easier!!
I think I got there by removing the addition disk from the volume group, shrinking the LV down to the extents allowed by the original disk (16G), re-adding the new disk to the VG, then finally growing the LV back to a size big enough to cover the etx4 file system (21G). Brought it back online and bingo, it mounted with all my data still there.
I'm now going to take a fresh copy of both VMDKs as backup. Then create a new partition on the first disk to allow me to use the remaining space. Add that new partition as a new PV, add the PV to the VG, expand the VG to use the full 50G and finally resize the ext4 system.
Then, if all goes well, implement a backup strategy to the production SAN so I don't have to go through this sh*t again if I screw up again.
And I thought LVM was meant to make life easier!!