(SOLVED) NTFS Partition reported as dirty by partclone



  • Hi,

    I've upgraded our CloneDeploy machine to 1.4.0 today and tried to upload an image from a freshly sysprepped Windows 10 machine.
    First 3 partitions (boot, efi, recovery) worked flawlessly, the fourth (system) partition was reported dirty by partclone and the task was aborted.

    Looking for similar thread I found one dated somewhere in 2016.
    In this thread it was mentioned that mixing legacy/EFI boot modes could be an issue.
    As booth modes lead to the same result on my machine, I cannot confirm this.

    Plugging the disk to another computer and checking the filesystem with checkdisk and fsutil resulted in the filesystem NOT being marked as dirty in any way.

    So the filesystems are clean but still reported as dirty by partclone.

    Any hint on how to continue with troubleshooting?

    Thanks
    Peter



  • can you attach the upload log



  • Sure, in the log there's also mentioned that Windows is hibernated.
    The system was sysprepped and shutdown, this machine has never seen hibernation mode.


    System Model: HP ProDesk 400 G5 MT

    ** Looking For Active Task For 10:e7:c6:XX:YY:ZZ.SN-EFP.44835739-DA0E-3CD8-A8CE-003A160330D8 **

    {"computerId":"false","task":"ond","taskId":null}

    ...... This Computer Was Not Found

    ** Looking For Active Task For 10:e7:c6:XX:YY:ZZ **

    {"computerId":"false","task":"ond","taskId":null}

    ...... This Computer Was Not Found

    ** Looking For Model Match Task For HP ProDesk 400 G5 MT **

    {"imageProfileId":null,"imageName":null,"imageProfileName":null}

    This Computer Is Not Registered. No Active Web Tasks Were Found For This Computer. Starting Registration.

    ** Displaying Boot Arguments **

    BOOT_IMAGE=/kernels/4.20.10x64 root=/dev/ram0 rw ramdisk_size=156000 web=http://192.168.99.3/clonedeploy/api/ClientImaging/ USER_TOKEN=38db8b70-8d38-422b-b02d-27695f033a39 consoleblank=0 MAC: 10:e7:c6:XX:YY:ZZ
    Linux client_console 4.20.10 #1 SMP Tue Feb 19 08:30:01 EST 2019 x86_64 GNU/Linux

    Boot Image Version: 1012

    ** Using On Demand Mode **

    ** Creating Active Task **

    image_name=EFP-W10-1809-2019-03 profile_id=11 pre_scripts="" post_scripts="" file_copy=False sysprep_tags="" task_completed_action="Reboot" image_type=Block skip_shrink_volumes=true compression_algorithm=gzip compression_level=-1 dp_id="1" computer_id=-155366

    ...... Success

    ** Checking Current Queue **

    ...... Complete

    ** Mounting SMB Share **

    ...... Connecting To Default

    ...... Success

    ** Looking For Hard Drive(s) **

    ...... Displaying Available Devices
    RO RA SSZ BSZ StartSec Size Device
    rw 256 512 1024 0 159744000 /dev/ram0
    rw 256 512 4096 0 128035676160 /dev/sda
    rw 256 512 4096 2048 523239424 /dev/sda1
    rw 256 512 4096 1024000 104857600 /dev/sda2
    rw 256 512 4096 1228800 16777216 /dev/sda3
    rw 256 512 4096 1261568 127389401088 /dev/sda4
    Disk /dev/ram0: 152.4 MiB, 159744000 bytes, 312000 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes

    Disk /dev/sda: 119.2 GiB, 128035676160 bytes, 250069680 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: BFA126E0-2DDD-4C86-8BB6-252D7C4025B8

    Device Start End Sectors Size Type
    /dev/sda1 2048 1023999 1021952 499M Windows recovery environment
    /dev/sda2 1024000 1228799 204800 100M EFI System
    /dev/sda3 1228800 1261567 32768 16M Microsoft reserved
    /dev/sda4 1261568 250068991 248807424 118.7G Microsoft basic data

    all_hard_drives: /dev/sda

    disk_label: gpt /dev/sda

    partitions: /dev/sda1
    /dev/sda2
    /dev/sda3
    /dev/sda4

    ...... Found Drive(s)

    ...... Drive(s): /dev/sda

    ** Removing All Files For Existing Image: EFP-W10-1809-2019-03 **

    ** Creating Image Schema **
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    Current NVRAM Entries
    BootCurrent: 0002
    Timeout: 0 seconds
    BootOrder: 0005,0006,0001,0002,0008,0004,0007,0003,0000
    Boot0000* Intel Corporation: Realtek PXE B01 D00 BBS(Network,Intel Corporation: Realtek PXE B01 D00,0x0)/PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/IPv4(0.0.0.00.0.0.0,0,0)......ISPH
    Boot0001 USB: PciRoot(0x0)/Pci(0x14,0x0)N.....YM....R,Y.....ISPH
    Boot0002* IPV4 Network - Realtek PCIe GBE Family Controller PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/MAC(10e7c6380277,0)/IPv4(0.0.0.00.0.0.0,0,0)N.....YM....R,Y.....ISPH
    Boot0003* SAMSUNG MZMPA128HMFU-000H1 BBS(HD,SAMSUNG MZMPA128HMFU-000H1 ,0x400)/PciRoot(0x0)/Pci(0x17,0x0)/Sata(0,65535,0)......ISPH
    Boot0004 USB: BBS(65535,,0x0)/PciRoot(0x0)/Pci(0x14,0x0)......ISPH
    Boot0005* Windows Boot Manager HD(2,GPT,d1df23db-39f0-4069-ba15-f40c08d2c7ab,0xfa000,0x32000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}...a....................ISPH
    Boot0006 SATA CD: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0,65535,0)N.....YM....R,Y..0..ISPH
    Boot0007 SATA CD: BBS(65535,,0x0)/PciRoot(0x0)/Pci(0x17,0x0)/Sata(0,65535,0)...0..ISPH
    Boot0008* IPV6 Network - Realtek PCIe GBE Family Controller PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)/MAC(10e7c6380277,0)/IPv6([::]:<->[::]:,0,0)N.....YM....R,Y.....ISPH
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    Calculating volume size for /dev/sda1
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda1) to image (/dev/null)
    Reading Super Block
    we need memory: 24228 bytes
    image head 4160, bitmap 15968, crc 4100 bytes
    Calculating bitmap... Please wait...
    used_size:13
    volume_size:524

    Partition Volume Size
    volume_size:524

    Partition Used Size
    used_size:13
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    Calculating volume size for /dev/sda2
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda2) to image (/dev/null)
    Reading Super Block
    we need memory: 30276 bytes
    image head 4160, bitmap 25600, crc 516 bytes
    Calculating bitmap... Please wait...
    used_size:32
    volume_size:106

    Partition Volume Size
    volume_size:106

    Partition Used Size
    used_size:32
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    Calculating volume size for /dev/sda3
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda3) to image (/dev/null)
    Reading Super Block
    get device size 16777216 by ioctl BLKGETSIZE64,
    we need memory: 8772 bytes
    image head 4160, bitmap 4096, crc 516 bytes
    Calculating bitmap... Please wait...
    used_size:17
    volume_size:17

    Partition Volume Size
    volume_size:17

    Partition Used Size
    used_size:17
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    Calculating volume size for /dev/sda4
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda4) to image (/dev/null)
    Reading Super Block
    we need memory: 3895876 bytes
    image head 4160, bitmap 3887616, crc 4100 bytes
    Calculating bitmap... Please wait...
    used_size:27988
    volume_size:127514

    Partition Volume Size
    volume_size:127514

    Partition Used Size
    used_size:27988

    ...... image_schema: {"harddrives": [ {"name":"/dev/sda","size":"250069680","table":"gpt","boot":"","lbs":"512","pbs":"512","guid":"BFA126E0-2DDD-4C86-8BB6-252D7C4025B8","active":"true","partitions": [ {"number":"1","start":"2048","end":"1023999","size":"1021952","volumesize":"499","type":"","usedmb":"12","fsType":"ntfs","fsid":"2700","uuid":"4480FCE880FCE0FA","guid":"14261510-B185-4671-96F7-A47B4F3A221E","active":"true","customsize":"","customsizeunit":"","forcefixedsize":"false","prefix":"","efibootloader":"","volumegroup": { } },{"number":"2","start":"1024000","end":"1228799","size":"204800","volumesize":"101","type":"","usedmb":"30","fsType":"vfat","fsid":"EF00","uuid":"FCFD-CD17","guid":"D1DF23DB-39F0-4069-BA15-F40C08D2C7AB","active":"true","customsize":"","customsizeunit":"","forcefixedsize":"false","prefix":"","efibootloader":"Windows Boot Manager#\EFI\Microsoft\Boot\bootmgfw.efi","volumegroup": { } },{"number":"3","start":"1228800","end":"1261567","size":"32768","volumesize":"16","type":"","usedmb":"16","fsType":"","fsid":"0C01","uuid":"","guid":"1F96288E-94A1-43F9-83FF-9590F28FEA78","active":"true","customsize":"","customsizeunit":"","forcefixedsize":"false","prefix":"","efibootloader":"","volumegroup": { } },{"number":"4","start":"1261568","end":"250068991","size":"248807424","volumesize":"121606","type":"","usedmb":"26691","fsType":"ntfs","fsid":"0700","uuid":"B0E4005FE40029E4","guid":"22A06916-5138-4EC8-A188-6B9E50D70B5B","active":"true","customsize":"","customsizeunit":"","forcefixedsize":"false","prefix":"","efibootloader":"","volumegroup": { } }] }] }

    ...... Complete

    ** Checking For Partition Table Types Before Upload On /dev/sda **
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    ...... gpt_status: present mbr_status: protective

    ** Saving MBR / GPT On /dev/sda **
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    ...... drive: checkPartStart: 2048 toSave: 2047
    2047+0 records in
    2047+0 records out
    1048064 bytes (1.0 MB, 1.0 MiB) copied, 0.0231943 s, 45.2 MB/s

    ...... Success

    ** Determining Partition Count On /dev/sda **

    ...... Found 4 Partition(s)

    ** Starting Image Upload For /dev/sda1 **

    ...... partition_number: 1 filesystem_type: ntfs

    ** Clearing Mounted Devices On /dev/sda1 **
    openHive(/mnt/ntfs/Windows/System32/config/SYSTEM) failed: No such file or directory, trying read-only
    openHive(/mnt/ntfs/Windows/System32/config/SYSTEM) in fallback RO-mode failed: No such file or directory
    reged version 0.1 140201, (c) Petter N Hagen
    Unable to open/read a hive, exiting..

    ...... Complete

    ...... Complete

    ...... lie_reporter 17 partclone & partclone.ntfs -B -c -s /dev/sda1 | gzip -1 -c > /storage/images/EFP-W10-1809-2019-03/hd0/part1.ntfs.gz
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda1) to image (-)
    Reading Super Block
    we need memory: 24228 bytes
    image head 4160, bitmap 15968, crc 4100 bytes
    Calculating bitmap... Please wait... done!
    File system: NTFS
    Device size: 523.2 MB = 127743 Blocks
    Space in use: 12.8 MB = 3125 Blocks
    Free Space: 510.4 MB = 124618 Blocks
    Block size: 4096 Byte
    Total block 127743
    Syncing... OK!
    Partclone successfully cloned the device (/dev/sda1) to the image (-)

    ** Starting Image Upload For /dev/sda2 **

    ...... partition_number: 2 filesystem_type: vfat

    ...... lie_reporter 17 partclone & partclone.fat -B -c -s /dev/sda2 | gzip -1 -c > /storage/images/EFP-W10-1809-2019-03/hd0/part2.fat.gz
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda2) to image (-)
    Reading Super Block
    we need memory: 30276 bytes
    image head 4160, bitmap 25600, crc 516 bytes
    Calculating bitmap... Please wait... done!
    File system: FAT32
    Device size: 104.9 MB = 204800 Blocks
    Space in use: 31.3 MB = 61044 Blocks
    Free Space: 73.6 MB = 143756 Blocks
    Block size: 512 Byte
    Total block 204800
    Syncing... OK!
    Partclone successfully cloned the device (/dev/sda2) to the image (-)

    ** Starting Image Upload For /dev/sda3 **

    ...... partition_number: 3 filesystem_type:
    The protective MBR's 0xEE partition is oversized! Auto-repairing.

    ...... lie_reporter 17 partclone & partclone.imager -B -c -s /dev/sda3 | gzip -1 -c > /storage/images/EFP-W10-1809-2019-03/hd0/part3.imager.gz
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda3) to image (-)
    Reading Super Block
    get device size 16777216 by ioctl BLKGETSIZE64,
    we need memory: 8772 bytes
    image head 4160, bitmap 4096, crc 516 bytes
    Calculating bitmap... Please wait... done!
    File system: raw
    Device size: 16.8 MB = 32768 Blocks
    Space in use: 16.8 MB = 32768 Blocks
    Free Space: 0 Byte = 0 Blocks
    Block size: 512 Byte
    Total block 32768
    Syncing... OK!
    Partclone successfully cloned the device (/dev/sda3) to the image (-)

    ** Starting Image Upload For /dev/sda4 **

    ...... partition_number: 4 filesystem_type: ntfs

    ** Clearing Mounted Devices On /dev/sda4 **
    Windows is hibernated, refused to mount.
    Falling back to read-only mount because the NTFS partition is in an
    unsafe state. Please resume and shutdown Windows fully (no hibernation
    or fast restarting.)
    Could not mount read-write, trying read-only
    openHive(/mnt/ntfs/Windows/System32/config/SYSTEM) failed: Read-only file system, trying read-only
    openHive(): read error: : Read-only file system
    reged version 0.1 140201, (c) Petter N Hagen
    Unable to open/read a hive, exiting..

    ...... Complete

    ...... Complete

    ** Removing PageFile **
    rm: cannot remove '/mnt/ntfs/Pagefile.sys': No such file or directory
    rm: cannot remove '/mnt/ntfs/pagefile.sys': Read-only file system

    ...... lie_reporter 17 partclone & partclone.ntfs -B -c -s /dev/sda4 | gzip -1 -c > /storage/images/EFP-W10-1809-2019-03/hd0/part4.ntfs.gz
    Partclone v0.2.90 http://partclone.org
    Starting to clone device (/dev/sda4) to image (-)
    Reading Super Block
    ntfsclone-ng.c: NTFS Volume '/dev/sda4' is scheduled for a check or it was shutdown
    uncleanly. Please boot Windows or fix it by fsck.

    shrunk_volumes:

    ** Closing Active Task **


    Regards
    Peter



  • Note to myself: read logs carefully, that's what they were made for.

    I'm currently uploading the image ...

    I've got it working by completely disabling hibernation mode before shutdown, which also removes the hibernation file.

    I can't remember doing anything different in the image creation process but never had this issue before.

    Edit:

    The issue is not with CloneDeploy or partclone, but with Windows 10 and the "wonderfull" fast startup feature.
    Running sysprep with /shutdown switch obviously doesn't do a shutdown but uses hibernation mode (if hibernation mode is available).

    Well, sorry for not RingTFM.

    Regards
    Peter