Skip to content

Remove lfs migrate recommendations for now #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+0 −17
Open

Remove lfs migrate recommendations for now #307

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions docs/guides/storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,6 @@ The `lfs getstripe <path>` command can be used to get information on the stripe
For directories and empty files `lfs setstripe --stripe-count <count> --stripe-size <size> <directory/file>` can be used to set the layout.

Striping settings on a directory are only applied to files added after the command is run.
Existing files retain their original layout unless explicitly changed using `lfs migrate <striping settings>`, which takes the same arguments as `lfs setstripe`.
The simplest way to have the correct layout is to copy to a directory with the correct layout.

!!! tip "A block size of 4MB gives good throughput, without being overly big..."
Expand All @@ -142,7 +141,6 @@ The simplest way to have the correct layout is to copy to a directory with the c
```

*Remember:* Settings applied with `lfs setstripe` only apply to files added to the directory after this command.
[Use `lfs migrate` to update the settings for existing files][ref-guides-storage-examples-lfs-migrate].

Lustre also supports composite layouts, switching from one layout to another at a given size `--component-end` (`-E`).
With it it is possible to create a Progressive file layout switching `--stripe-count` (`-c`), `--stripe-size` (`-S`), so that fewer locks are required for smaller files, but load is distributed for larger files.
Expand All @@ -152,21 +150,6 @@ With it it is possible to create a Progressive file layout switching `--stripe-c
lfs setstripe --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count 32 --stripe-size 4M <base_dir>
```

[](){#ref-guides-storage-examples-lfs-migrate}
!!! example "Updating settings for existing files"
While `lfs setstripe` applies to newly created files, `lfs migrate` can be used to re-layout existing files.
For example, to set the recommended settings above on an existing file:
```bash
lfs migrate --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count 32 --stripe-size 4M <file>
```

Alternatively, to migrate all files recursively in a directory:
```bash
lfs find --type file <base_dir> | xargs lfs migrate --verbose --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count 32 --stripe-size 4M
```
The `--verbose` flag makes `lfs migrate` print the path of each file after the file has been migrated.
Also note the use of `lfs find` instead of regular `find` as `lfs` can more efficiently retrieve the list of files recursively.

### Iopsstor vs Capstor

[Iopsstor][ref-alps-iopsstor] uses SSD as OST, thus random access is quick, and the performance of the single OST is high.
Expand Down