Skip to content

Conversation

cehbrecht
Copy link
Collaborator

@cehbrecht cehbrecht commented Mar 21, 2021

Overview

This PR adds a new server option storage_copy_function. You can choose the copy_function in FileStorage when copying outputs to the destination folder: copy, move or link. Default=copy (as it was already).

Motivation: when having large output files we don't want to copy them ... preferred method is link making a hardlink using os.link. move would be an alternative using shutil.move. Fallback is always copy with shutil.copy2.

Related Issue / Discussion

Additional Information

The link was also implemented in pywps 3.x:

elif not self._samefile(output.value,outFile):
# TODO: dirty hack to avoid copy time
try:
LOGGER.debug("link output: from=%s, to=%s", os.path.abspath(output.value), outFile)
os.link(os.path.abspath(output.value), outFile)
except:
LOGGER.warn("failed to link output")
COPY(os.path.abspath(output.value), outFile)

Contribution Agreement

(as per https://github.com/geopython/pywps/blob/master/CONTRIBUTING.rst#contributions-and-licensing)

  • I'd like to contribute [feature X|bugfix Y|docs|something else] to PyWPS. I confirm that my contributions to PyWPS will be compatible with the PyWPS license guidelines at the time of contribution.
  • I have already previously agreed to the PyWPS Contributions and Licensing Guidelines

@cehbrecht cehbrecht self-assigned this Mar 21, 2021
@coveralls
Copy link

coveralls commented Mar 21, 2021

Coverage Status

Coverage remained the same at 0.0% when pulling 5560952 on cehbrecht:storage-copy-function into 0303a6e on geopython:pywps-4.4.

@cehbrecht
Copy link
Collaborator Author

ping @Zeitsperre @huard

@cehbrecht cehbrecht merged commit 51da703 into geopython:pywps-4.4 Mar 21, 2021
@cehbrecht cehbrecht deleted the storage-copy-function branch March 21, 2021 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants