Upload Module

The upload module used to send the data to ingest.

pacifica.cli.upload.build_file_list_from_args(file_list, followlinks)

Build a file list from args passed on cmdline.

pacifica.cli.upload.check(status)

Check the status since it’s complicated.

pacifica.cli.upload.check_okay(status)

Check if the status is something wrong.

pacifica.cli.upload.determine_sizes(md_update, args)

Return the sizes of the tar, tar_in_tar and overall content length.

pacifica.cli.upload.fake_uploader(rfd, content_length)

Fake the upload by reading all the content then returning.

pacifica.cli.upload.generate_names_from_dir(dirpath, followlinks)

Generate a file list from a dirpath.

pacifica.cli.upload.get_size_of_tar(md_update, args)

Get the size of the tar file, no tarintar.

pacifica.cli.upload.get_size_of_tar_in_tar(md_update, args, tar_size)

Return the content-length for the upload.

pacifica.cli.upload.invoke_uploader(md_update, rfd, content_length)

Invoke the uploader code to actually upload.

pacifica.cli.upload.perform_upload(md_update, args, content_length, tar_size)

Setup threads and perform the upload.

pacifica.cli.upload.pipefds()

Setup pipe but return file objects instead.

pacifica.cli.upload.save_local(rfd, wfd, save_filename, compressor)

Save the bytes from rfd to args.savelocal and wfd.

pacifica.cli.upload.setup_bundler(wfd, md_update, args, wthreads)

Setup the bundler or local retry if passed.

pacifica.cli.upload.setup_chain_thread(pipes, args, func, wthreads, doit)

Setup a local thread if we doit using func.

pacifica.cli.upload.tar_in_tar(rfd, wfd, md_update, bundle_size)

Generate another bundler and wrap rfd in that tar.

pacifica.cli.upload.upload_files_from_args(file_list, followlinks, prefix)

Generate a files structure required by bundler.

pacifica.cli.upload.upload_main(md_update, args)

Main upload method.

pacifica.cli.upload.wait_for_upload(args, jobid, up_obj)

Wait (or not) for the jobid to complete the ingest process.