parted: Resizing not working

[MarcusCaepio]

SUMMARY

Hi all,
I am trying to resize a partition (/dev/sda3), which is a physical volume of lvm. The resize part according to the docs (where btw is a typo in the resize example) just does nothing:

ISSUE TYPE

• Bug Report

COMPONENT NAME

parted

ANSIBLE VERSION

ansible 2.10.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]

STEPS TO REPRODUCE

• Create a VM in e.g. VMware with Ubuntu 20.04 and a HDD of 15GB
• Install it with LVM, where /dev/sda3 is a PV
• Resize the HDD in VMware to e.g. 25GB, reboot or rescan the HDD in Ubuntu. /dev/sda is 25GB now

Disk /dev/sda: 25 GiB, 26843545600 bytes, 52428800 sectors
Disk model: Virtual disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt

Device       Start      End  Sectors Size Type
/dev/sda1     2048     4095     2048   1M BIOS boot
/dev/sda2     4096  2101247  2097152   1G Linux filesystem
/dev/sda3  2101248 33552383 31451136  15G Linux filesystem

• try to resize /dev/sda3 with Ansible to the max available space

  - name: Read device information (always use unit when probing)
    community.general.parted:
      device: /dev/sda
      unit: MiB
    register: sda_info
    
  - name: Extend an existing partition to fill all available space
    community.general.parted:
      device: /dev/sda
      number: "{{ sda_info.partitions | length }}"
      part_end: "100%"
      resize: true

EXPECTED RESULTS

/dev/sda3 should be resized to the max given space

ACTUAL RESULTS

Nothing happens

ok: [xxx] => {
    "changed": false,
    "disk": {
        "dev": "/dev/sda",
        "logical_block": 512,
        "model": "VMware Virtual disk",
        "physical_block": 512,
        "size": 26214400.0,
        "table": "gpt",
        "unit": "kib"
    },
    "invocation": {
        "module_args": {
            "align": "optimal",
            "device": "/dev/sda",
            "flags": null,
            "fs_type": null,
            "label": "msdos",
            "name": null,
            "number": 3,
            "part_end": "100%",
            "part_start": "0%",
            "part_type": "primary",
            "resize": true,
            "state": "info",
            "unit": "KiB"
        }
    },
    "partitions": [
        {
            "begin": 1024.0,
            "end": 2048.0,
            "flags": [
                "bios_grub"
            ],
            "fstype": "",
            "name": "",
            "num": 1,
            "size": 1024.0,
            "unit": "kib"
        },
        {
            "begin": 2048.0,
            "end": 1050624.0,
            "flags": [],
            "fstype": "ext4",
            "name": "",
            "num": 2,
            "size": 1048576.0,
            "unit": "kib"
        },
        {
            "begin": 1050624.0,
            "end": 16776192.0,
            "flags": [],
            "fstype": "",
            "name": "",
            "num": 3,
            "size": 15725568.0,
            "unit": "kib"
        }
    ],
    "script": "unit 'KiB' print"
}

Thanks,
Marcus

[ansibullbot]

Files identified in the description:

• plugins/modules/system/parted.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @ColOfAbRiX @jake2184 @rosowiecki
click here for bot help

[jake2184]

You're missing state: present. The default value for state is info.

This is an omission in the docs (my fault), will fix now. Thanks!

[jake2184]

It's also worth pointing out if you're using GPT tables you might need to have checks around this module to fix the GPT missing at the end of a disk after a resize.

See #1461

[MarcusCaepio]

Thanks for the hint. But it turns out, that parted is not able to resize a partition, which is in use.

fatal: [xxx]: FAILED! => {
    "changed": false,
    "err": "Error: Partition(s) on /dev/sda are being used.\n",
    "invocation": {
        "module_args": {
            "align": "optimal",
            "device": "/dev/sda",
            "flags": null,
            "fs_type": null,
            "label": "msdos",
            "name": null,
            "number": 3,
            "part_end": "100%",
            "part_start": "0%",
            "part_type": "primary",
            "resize": true,
            "state": "present",
            "unit": "KiB"
        }
    },
    "msg": "Error while running parted script: /usr/sbin/parted -s -m -a optimal /dev/sda -- unit KiB mklabel msdos mkpart primary 0% 100% resizepart 3 100%",
    "out": "",
    "rc": 1
}

Is this a limitation of parted? I normally use cfdisk for resizing partitions, which is no problem.

[jake2184]

I hit this when testing after your comment, I'm a bit surprised here - I'm using this module to regularly resize in-use DOS partitions.
I wonder if that's a GPT issue. Put bluntly, I don't think parted (and hence this module) has the best support for GPT tables.

[MarcusCaepio]

It seems not to a problem of parted itself, but of the module in any way. I can resize the partition easily with parted on CLI:

root@localhost:~# fdisk -l | grep /dev/sda
GPT PMBR size mismatch (33554431 != 52428799) will be corrected by write.
Disk /dev/sda: 25 GiB, 26843545600 bytes, 52428800 sectors
/dev/sda1     2048     4095     2048   1M BIOS boot
/dev/sda2     4096  2101247  2097152   1G Linux filesystem
/dev/sda3  2101248 33552383 31451136  15G Linux filesystem

root@localhost:~# parted -s /dev/sda resizepart 3 100%

root@localhost:~# fdisk -l | grep /dev/sda
Disk /dev/sda: 25 GiB, 26843545600 bytes, 52428800 sectors
/dev/sda1     2048     4095     2048   1M BIOS boot
/dev/sda2     4096  2101247  2097152   1G Linux filesystem
/dev/sda3  2101248 52428766 50327519  24G Linux filesystem

Is the module using "resize" or "resizepart" ? Because resize was removed after parted v. 2.4

[jake2184]

You can see the exact command run from the error message you posted:

Error while running parted script: /usr/sbin/parted -s -m -a optimal /dev/sda -- unit KiB mklabel msdos mkpart primary 0% 100% resizepart 3 100%"

The mklabel msdos seems dubious - I think the root cause is that the module is trying to create a completely new partition table , and rightfully throwing a warning that some partitions are in use.

The default is label: msdos. Can you try again explicitly with label: gpt. Sorry, should have spotted that earlier.

[MarcusCaepio]

Sorry, should have spotted that earlier.

Np, I am glad you responded so fast. :)

Giving the label works.
So for the docs:

  - name: Read device information (always use unit when probing)
    community.general.parted:
      device: /dev/sda
      unit: MiB
    register: sda_info

  - name: Extend an existing partition to fill all available space
    community.general.parted:
      device: /dev/sda
      number: "{{ sda_info.partitions | length }}"
      label: gpt
      unit: MiB
      part_end: "100%"
      state: present
      resize: true

But wouldn't it be nice, when the module automatically sets the correct label on "resize: true" ? And imho, "resize: true" automatically implements "state: present", so I think it shouldn't has to be to set additionally :)

[jake2184]

The docs are correct as they are now - GPT is a non-default label, so it's understandable you have to declare it explicitly.

Whilst I agree with your last comment about it being set automatically, it requires editing logic of the module outside of the scope I've done before (I only did the resize extension) so I would like another maintainer to agree with that first.

[sjpb]

I've also hit this, but with a disk without a label so that's not the problem. Ansible-generated command was:

Error while running parted script: /sbin/parted -s -m -a optimal /dev/sdb -- resizepart 1 100%

fails with

    "err": "Warning: Partition /dev/sdb1 is being used. Are you sure you want to continue?\n",

Trying the same command manually also fails - I think the actual problem is the -s flag as with this parted doesn't query the user at all, it just prints the above prompt and exits.

[sjpb]

Horribly it looks like it might need an undocumented flag to fix: https://stackoverflow.com/a/58840066/916373

[sebastianmacarescu]

Any workaround for this?

[jake2184]

Sorry, I missed this.
I haven't encountered this issue before, and I'm using this module every day to resize disks with partitions that are in use.

Are you able to provide a reproducible scenario I can test with?

[sebastianmacarescu]

I've tried to resize the root volume volume of AWS t3.micro (nvme based), Ubuntu 18.04 ami.
I ended up with this workaround:

- name: Extend partition to fill all available space
  ansible.builtin.command: "/sbin/parted ---pretend-input-tty /dev/{{ device }} resizepart {{ partition_num }} yes 100%"
  become: yes

[thywyn]

This reproducible on the centos 8 stream image, trying to resize the root partition.
Kernel: 4.18.0-348.2.1.el8_5.x86_64
parted (GNU parted): 3.2
ansible 2.9.24
community.general: 4.1.0
python: 3.9.6

The '-s' assumes no input even though the command expects a 'Yes' to confirm, so nothing happens. Could not confirm sebastianmacarescu's solution on this image but the solution from the below link worked:
https://stackoverflow.com/questions/52509644/how-do-i-get-over-parted-confirmation-request-in-a-script/58840066#58840066
echo -e "resizepart\n1\nYes\n100%\n"| sudo parted /dev/vda ---pretend-input-tty

[jake2184]

So I'm able to replicate this, but I'm not sure there is a simple fix.

You're trying to use parted to alter GPT partition tables (that wasn't clear from the context), which parted just doesn't have very good support for - as per the previous comments in this issue, and the fact it's trying to get input in script mode.

The stated workarounds might work, but they change the nature of the module, meaning it doesn't run in script-mode (currently hard-coded), and relies on an un-documented flag. I don't think the solution is to include that in the module code. See #1461 for similar comments.

I recommend that you use a tool like sgdisk to manipulate GPT partitions - it's already installed on the stream image, and is a tool designed specifically for scripting changes to GPT disks.

[ekrimi]

@sebastianmacarescu thank you for the workaround it works perfectly

[sjpb]

@jake2184 is it appropriate to add a note to the docs for community.general.parted then? As someone who wasn't even aware that a "GPT partition table" was a thing, it is definitely not obvious whether resize: true is going to work.

[jake2184]

Yes, that's a fair point and one I meant to add earlier.

The most frustrating thing I found (and why I use this module in the first place) is the lack of an Ansible module that wraps around GPT tools!

I'll update the docs

[anlag]

No help ultimately required, see edit at the end.

So I'm running into problems resizing a partition with msdos label, hard to be sure whether it's related to this but it's certainly similar. The task is as follows:

- name: Create data partition
  parted:
    device: /dev/sdb
    number: 1
    state: present
    label: msdos
    resize: yes

I did not initially have label: msdos in there, it however makes no difference.

Here's the output:

TASK [rhel8_base : Create data partition] ********************************************************************************************************************
fatal: [gc0altest01]: FAILED! => {"changed": false, "err": "Warning: Partition /dev/sdb1 is being used. Are you sure you want to continue?\n", "msg": "Error while running parted script: /sbin/parted -s -m -a optimal /dev/sdb -- resizepart 1 100%", "out": "", "rc": 1}

The disk in question, parted -l output:

Model: Google PersistentDisk (scsi)
Disk /dev/sdb: 16.1GB
Sector size (logical/physical): 512B/4096B
**Partition Table: msdos**
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  14.0GB  14.0GB  primary  xfs

Now, it should be said using the parted command directly also does not work:

[root@gc0altest01 ~]# /sbin/parted -s -m -a optimal /dev/sdb -- resizepart 1 100%
Warning: Partition /dev/sdb1 is being used. Are you sure you want to continue?

I suppose based on that, you could argue it's an issue with parted rather than with the module, and if it is indeed an upstream bug and there's nothing to be done here, then fair enough. I guess I'm holding out hope there might be a way to make this work nicely with the module.

Admittedly, the workaround above does work:

[root@gc0altest01 ~]# echo -e "resizepart\n1\nYes\n100%\n"| sudo parted /dev/sdb ---pretend-input-tty
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) resizepart                                                       
Partition number? 1                                                       
Warning: Partition /dev/sdb1 is being used. Are you sure you want to continue?
Yes/No? Yes                                                               
End?  [16.1GB]? 100%                                                      
(parted)                                                                  
(parted)                                                                  
Information: You may need to update /etc/fstab.

Before going down that route, I think I'd rather create the filesystem directly on the block device and cut the partition step entirely. Still, I'd appreciate any help here.

--
EDIT: This VM lives in Google Cloud, and subsequent to posting I found the following recommendation in their docs:

You can use any file format that you need, but we recommend a single ext4 file system without a partition table. You can increase the size of your disk later without having to modify disk partitions.

https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting

I will therefore move ahead without creating any partition and so don't need any help on this issue. Nevertheless I will leave this comment in the event it helps anyone else.

[ansibullbot]

Files identified in the description:

• plugins/modules/parted.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[Fail-Unsafe]

Hi, is there any kind of progress on this issue? I would really love to use the parted module with my Ubuntu servers that have GPT partition tables.

My workaround is to use the command module to execute the command growpart /dev/sda 2 and then use the filesystem module to resize the ext4 filesystem. Is there are any way to wrap the growpart command inside the parted module?

[jake2184]

I'm not sure there is an identifiable issue with the module here, other than that the underlying binary doesn't have good support for GPT tables.

This module is purely a wrapper around parted, adding in another binary is out of scope. As stated above, when I last checked a partitioning module which properly supports GPT tables is missing in the main Ansible collections, but extending parted isn't the correct way to solve that.

[oldmanhere]

This issue is solved on rhel9 to extend LVM partitions with GPT by replace in plugins/modules/parted.py the line 573:
command = "%s -s -m %s %s -- %s" % (parted_exec, align_option, device, script)
by
command = "%s -s -f -m %s %s -- %s" % (parted_exec, align_option, device, script)

The man command for parted say on my OS :
-f, --fix
automatically answer "fix" to exceptions in script mode

I don't know if this option exist on other OS
You can read this: https://www.mail-archive.com/parted-devel@alioth-lists.debian.net/msg00369.html

[puntopaz]

Just for reference, this is another possible workaround using expect module:

#----------------------------------------------------
- name: Extending existing partition to fill all available space
  expect:
    command: "/usr/sbin/parted -m -a optimal {{ device }} -- resizepart {{ partition_num }} 100%"
    #timeout: 10
    echo: yes
    responses:
      # Warning: Not all of the space available to /dev/nvme0n1 appears to be used, you can fix the GPT to use all of the space (an extra 46137344 blocks) or continue with the current setting?
      "Warning: Not all of the space available(.*)": "Fix"
      # Warning: Partition /dev/nvme0n1p1 is being used. Are you sure you want to continue?
      "Warning: Partition [\\w/]+ is being used(.*)": "Yes"
      # Parted standard questions
      "Partition number?": "{{ partition_num }}"
      "End?(.*)": "100%"
  become: yes
  delegate_to: "{{ instance_ip_address }}"
  register: parted_resize_info
  when: partition_num != ''
#----------------------------------------------------
- debug:
    var: parted_resize_info