Accessing to USB drive from boot initrd img



  • Hi all,

    I am looking to get to get access to the USB Drive from which the initrd image is extracted and mounted. I have already converted the USB stick into NTFS/syslinux bootable USB device, which will allow the USB to handle large files. At this point I see the drive as the Linux kernel is loading, but it doesn't show up with fdisk -l. It looks like it might be disabled somehow. If this is possible. I would like to get this enabled for deployment. My goal script a check for an i.e. "Image folder" that contains an image, and mounts or symlink it as /Storage/ instead of the SMB share. This would allow a user to deploy the image from the USB device to the hard drive. This would be ideal for areas that only need one machine done at time with very limited bandwidth. If you could give me some help and direction on this project it would be appreciated.

    Thank you,

    Tim



  • Just noticed the new 4.5 kernel, and the drive is showing up now... ha!



  • This is something I have also been giving some thought to lately. My idea of how it would work is nearly identical to what you are trying. I would be very interested to hear your results as well as any documentation you could provide. One issue I can foresee is trying to prevent CloneDeploy from trying to image the usb drive.



  • Here is the scripts I have setup in my environment, and it looks like it does the trick. I use the readlink command to verify the drive is a USB device.

    I noticed the image check in the mount_smb() function wasn't really check for the correct folder. The variable for the image name was set as $imgName, and it looks like the correct variable is $image_name.

    Modified the following scripts to provide USB deployment:

    [h][b]** Added usb_deployment function to cd_global_functions script **[/b][/h]

    [code]
    function usb_deployment()
    {
    is_usbdeployment='false'
    log " ** Checking for USB Image for current deployment **" "display"
    mkdir /storage &>> $CLIENT_LOG
    usbdrives=()
    drivelist=lsblk -ipdno KNAME
    for drive in $drivelist; do
    driveName=$(echo "$drive" | sed 's//dev//''/g')
    driveLink=$(readlink /sys/block/$driveName)
    if echo "$driveLink" | grep -q -m 1 "usb"; then
    echo "Found USB Drive: $drive"
    usbdrives+=("$drive")
    fi
    done
    for usbhdd in $usbdrives; do
    echo "Checking USB drive: $usbhdd"
    partlist=lsblk "$usbhdd" -ln -o NAME
    i=0
    for part in $partlist; do
    if [ $i != 0 ]; then
    echo " ------> Mounting Partition: $parts"
    mount -t ntfs -o ro "/dev/$part" /storage
    if [ "$?" == 0 ]; then
    if [ -e /storage/images/$image_name ]; then
    is_usbdeployment='true'
    break
    else
    umount "/dev/$part"
    is_usbdeployment='false'
    fi
    else
    echo " ------> Failed to mount partition."
    is_usbdeployment='false'
    fi
    fi
    i=$(($i + 1))
    done
    if [ "$is_usbdeployment" == 'true' ]; then
    break
    fi
    done
    if [ "$is_usbdeployment" == 'true' ]; then
    echo " ......USB images found."
    else
    echo " ......No USB deployment images were found."
    fi
    echo
    sleep 2
    }
    [/code]

    [h][b]** updated cd_push script**[/b][/h]

    Edits:

    [b]usb_deployment

    if [ "$foundImg" == 'false' ]; then
    mount_smb
    fi
    [/b]

    [code]
    function main()
    {

    log_boot_args

    is_usbdeployment='false'

    if [ "$is_ondemand" = "true" ]; then
    log " ** Using On Demand Mode ** "
    for arg in "$ondemand_arguments"; do case "$arg" in =) eval "$arg"; log "$arg"; esac; done
    if [ "$task" == 'push' ] && [ "$multicast" != 'true' ]; then
    usb_deployment
    fi
    elif [ "$task" = "clobber" ]; then
    log " ** Using Clobber Mode ** "
    clobber_args=$($curlAuth --data "mac=$mac&objectId=$imageProfileId&task=push" "${web}GetOnDemandArguments" $curlEnd)
    for arg in "$clobber_args"; do case "$arg" in =) eval "$arg"; log "$arg"; esac; done
    if [ "$computer_name" = "" ] || [ "$computer_name" = "null" ]; then
    if [ "$name_prompt" = "true" ]; then
    echo "This Computer Was Not Found In The Database, Enter A Name For It Now:"
    echo "Leave Blank To Skip Computer Renaming"
    echo
    read computer_name
    echo
    if [ -n "$computer_name" ]; then
    computer_name=$computer_name
    fi
    fi
    fi
    else
    checkin
    fi

    if [ "$is_ondemand" != "true" ]; then
    if [ "$task" = "push" ] || [ "$task" = "permanent_push" ]; then
    if [ "$multicast" = "true" ]; then
    $curlAuth --data "computerId=$computer_id" "${web}UpdateStatusInProgress" $curlEnd
    else
    usb_deployment
    if [ "$is_usbdeployment" == 'false' ]; then
    log " ** Checking Current Queue ** " "display"
    while true; do
    queue_status=$($curlAuth --data "computerId=$computer_id" "${web}CheckQueue" $curlEnd)
    if [ "$(parse_json "$queue_status" .Result)" = "true" ]; then
    break
    else
    echo "** Queue Is Full, Waiting For Open Slot ** "
    echo " ...... Current Position $(parse_json "$queue_status" .Position)"
    sleep 5
    fi
    done
    log " ...... Complete" "display"
    echo
    fi
    fi
    fi
    fi

    sleep 2

    if [ "$is_usbdeployment" == 'false' ]; then
    mount_smb
    fi

    process_hard_drives

    checkout
    }

    [/code]

    [h][b]** Fixes to mount_smb() function in cd_global_functions **[/b][/h]

    edits:
    [b]cd /storage/images/$image_name;[/b]

    [code]

    function mount_smb()
    {
    log " ** Mounting SMB Share **" "display"
    mkdir /storage &>> $CLIENT_LOG
    smbInfo=$($curlAuth --data "dpId=$dp_id&task=$task" "${web}DistributionPoint" $curlEnd)
    for smbVersion in 3.0 2.1 2.0 1.0; do
    mount -t cifs "$(parse_json "$smbInfo" .SharePath)" /storage -o user="$(parse_json "$smbInfo" .Username)",pass="$(parse_json "$smbInfo" .Password)",dom="$(parse_json "$smbInfo" .Domain)",vers=$smbVersion 2>/tmp/mntstat
    if [ ! $? = "0" ]; then
    cat /tmp/mntstat >> $CLIENT_LOG
    errormsg=$(cat /tmp/mntstat)

    	else
    		log " ...... Success" "display"
    		cd /storage/images/$image_name;
    		if [ "$?" != "0" ]; then
    			error "Could Not Change Directory To /images/$image_name Check Permissions"
    		fi
    		break
    	fi
    	if [ "$smbVersion" = "1.0" ]; then
    		error "Could Not Mount SMB Share"
    	fi
    done
    echo
    sleep 2
    

    }

    [/code]



  • [h][b]** building USB with NTFS **[/b][/h]

    Download Rufus USB Tool: https://rufus.akeo.ie/

    Open Rufus and select the client.iso provided by clonedeploy.
    [attachment:576c656250e82]
    Once the ISO file is loaded, select file system type: NTFS and click start.
    [attachment:576c6602e89be]
    Next apply USB Generator files changes to the USB drive. (For help - http://clonedeploy.org/docs/client-boot-method/)
    [attachment:576c67cb2ebbc]
    Create new folders call images and resources on the root of the USB drive.
    [attachment:576c6820e4ef4]
    Download the latest version of Syslinux: (used v6.0.3) - https://www.kernel.org/pub/linux/utils/boot/syslinux
    Replace the following files in Syslinux on your USB drive:
    bois\core\isolinux.bin to Syslinux
    bios\com32\elflink\ldlinux\ldlinux.c32 to Sysliux
    Run utils\win32\makeboot.bat on USB Drive, run this as administrator.
    Copy the deployment image and other resources the correct corresponding folders on the USB drive



  • Fantastic. Thanks for all your work on this. Can't wait to try it out, and get it added into the next release.



  • One change I will probably make is to check for the usb drive before checking the queue and skip the queue if it exists. The queue is there specifically to throttle the number concurrent imaging processes taking place, don't really see the need to wait for usb. Most people are probably using on demand if using usb, but you never know.



  • Good idea. I updated the previous post to reflex this. Added an extra if statement after the checkin, "if [ "$foundImg" == 'false' ]; then" and moved the usb_deployment function to the front of the main function in cd_push.



  • I might have to change something with the scripts. Hopefully you can help me out with this. I cant seem to find where $Image_name is set. After installing the fresh copy of your new release. I noticed that $image_name variable isn't set by the time the usb_deployment function is run. Is there a way I could query for this value?



  • It should be, that process hasn't changed.

    When using a task started from the web it is set from the checkin function from these 2 lines, pulled from the web service
    [code]checkInStatus=$curlAuth --data "computerMac=$mac" "${web}CheckIn" $curlEnd
    for arg in "parse_json "$checkInStatus" .TaskArguments"; do case "$arg" in =) eval "$arg"; log "$arg"; esac; done[/code]

    On demand is similar
    [code]for arg in "$ondemand_arguments"; do case "$arg" in =) eval "$arg"; log "$arg"; esac; done[/code]
    in each push and pull script, the original service is called from the ond script and passed in



  • Oh i see, yea you moved it too high.



  • ah! Thanks I will update this.



  • I discovered a minor issue with this. This does not work with EFI secure boot. EFI can only boot to fat partitions, Rufus puts a special efi partition at the end of the drive when ntfs is selected, it is unsigned and therefore breaks secure boot. If we change Rufus to use a fat32 partition secure boot will work but then the image will be too big for the 4GB fat32 file size limit. Ideally we need to find a way to keep the boot part as fat32 and create an additional ntfs partition for the image.



  • Can partimg split the disk image into 4gb parts? Also i make a fat32 60mb partition on and run the syslinux bat and the make the rest of the drive a ntfs. This allows the efi to work.



  • Very Nice. Looking to do the same thing, but in winPE. Did this get added into any release?
    I am still on 1.2 and wont have time to modify the code for PE any time soon..



  • This has not yet been added. It is currently in the works for Linux and should be available in the next release. WinPE has not been started.