Conversation
There was a problem hiding this comment.
Pull request overview
Introduces documentation for the new unified storage.yaml cache configuration, and marks the legacy storage.config and volume.config formats as deprecated. It also updates the admin config index and documents per-volume avg_obj_size and fragment_size overrides.
Changes:
- Add comprehensive
storage.yamldocumentation including schema (spans,volumes,volumes.spans), allocation rules, and migration/backwards compatibility information. - Mark
storage.configandvolume.configas deprecated in favor ofstorage.yamland cross-link them appropriately. - Update the admin “Configuration Files” index to include
storage.yamland to reflect the new deprecation status ofstorage.configandvolume.config.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
doc/admin-guide/files/storage.yaml.en.rst |
New, full reference and examples for storage.yaml, covering storage spans, volume definitions (including avg_obj_size and fragment_size), allocation behavior, and compatibility with existing storage.config/volume.config. |
doc/admin-guide/files/storage.config.en.rst |
Adds an .. important:: block to mark storage.config as deprecated in favor of storage.yaml. |
doc/admin-guide/files/volume.config.en.rst |
Adds an .. important:: block to mark volume.config as deprecated in favor of storage.yaml. |
doc/admin-guide/files/index.en.rst |
Registers storage.yaml in the configuration files toctree and updates one-line descriptions for storage.config, storage.yaml, and volume.config to reflect the new deprecations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d3a7be1 to
e3e54b8
Compare
Co-authored-by: Alan M. Carroll <amc@apache.org>
e3e54b8 to
ba0370d
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Feature comparison: storage.config + volume.config vs storage.yaml I compared the documented features in the existing config files against the new
Items 1-3 are the most important -- users will hit these in practice. 4 and 5 are nice-to-have. Otherwise the docs look thorough. The examples are clear, the backwards compatibility section is helpful, and the |
bryancall
left a comment
There was a problem hiding this comment.
Please see my comment on the PR about the changes in the documentation
Add an example with 64G
Updated
There is a warning after the key descriptions
It's already mentioned exactly same as storage.config ( it was just copied )
Copied the sentence from volume.config |
|
[approve ci centos] |
| | fragment_size | integer | Overrides the global :ts:cv:`proxy.config.cache.target_fragment_size` configuration for this volume. | | ||
| | | | This allows for a smaller, or larger, fragment size for a particular volume. This may be useful | | ||
| | | | together with ``avg_obj_size`` as well, since a larger fragment size could reduce the number of | | ||
| | | | directory entries needed for a large object. Note that this setting has a maximmum value of 4MB. | |
There was a problem hiding this comment.
Typo in this table cell: "maximmum" should be "maximum".
| | | | directory entries needed for a large object. Note that this setting has a maximmum value of 4MB. | | |
| | | | directory entries needed for a large object. Note that this setting has a maximum value of 4MB. | |
| which will effectively clear most of the cache. This can be a problem when drives fail and a system | ||
| reboot causes the path names to change. | ||
|
|
||
| The :arg:`name` option can be used to create a fixed string that an administrator can use to keep the |
There was a problem hiding this comment.
The assignment-table paragraph refers to the ":arg:name option" as controlling the assignment hash seed, but earlier the document defines hash_seed as the key for isolating lookups from path changes. This looks inconsistent with the key definitions (and with storage.config where id= seeds the assignment table). Consider updating this to refer to the hash_seed key (and use a role consistent with other YAML-key references, e.g., hash_seed / :code:).
| The :arg:`name` option can be used to create a fixed string that an administrator can use to keep the | |
| The :code:`hash_seed` key can be used to create a fixed string that an administrator can use to keep the |
| /dev/disk2 volume=3 # storage.config | ||
| volume=3 scheme=http size=512 # volume.config | ||
|
|
||
| The corresponding configuration would be | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| cache: | ||
| spans: | ||
| - name: disk.2 | ||
| path: /dev/disk2 | ||
| volumes: | ||
| - id: 3 | ||
| spans: | ||
| - use: disk.2 | ||
| size: 512 | ||
|
|
||
| Because volume sizes that are percentages are computed on span storage not already explicitly allocated, this will | ||
| leave none of "disk.2" for such allocation and therefore "disk.2" will be used only by volume "1". Note this | ||
| configuration is more flexible. If it was useful to have two linear volumes, each using exclusively half of the |
There was a problem hiding this comment.
Backwards-compatibility example appears inconsistent: it defines volume id: 3 but the explanation says the span will be used only by volume "1". Also, the example uses size: 512 (bytes per earlier text) which conflicts with the 128MB allocation granularity described later and is likely meant to be 512M (matching the legacy volume.config example).
| Defines cache space usage by individual protocols. (Deprecated in favor of :file:`storage.yaml`) | ||
|
|
||
| :doc:`jsonrpc.yaml.en` | ||
| Defines some of the configurable arguments of the jsonrpc endpoint. |
There was a problem hiding this comment.
These two definition entries are indented one extra space compared to the rest of the list, which is likely to render oddly in Sphinx. Align the indentation of the volume.config.en and jsonrpc.yaml.en description lines with the other entries (4 spaces).
| Defines cache space usage by individual protocols. (Deprecated in favor of :file:`storage.yaml`) | |
| :doc:`jsonrpc.yaml.en` | |
| Defines some of the configurable arguments of the jsonrpc endpoint. | |
| Defines cache space usage by individual protocols. (Deprecated in favor of :file:`storage.yaml`) | |
| :doc:`jsonrpc.yaml.en` | |
| Defines some of the configurable arguments of the jsonrpc endpoint. |
| cache: | ||
| spans: | ||
| - name: disk | ||
| path: "/dev/sdb" | ||
| - name: ram.1 | ||
| path: "/dev/ram.1" | ||
| - name: ram.2 | ||
| path: "/dev/ram.2" | ||
| volumes: | ||
| - id: 1 |
There was a problem hiding this comment.
The YAML indentation in this example is inconsistent with the other snippets (extra leading spaces before spans: / volumes:). Keeping indentation consistent makes the examples easier to copy/paste and compare.
Revive #11000 for the config project. Diffs from #11000 are
storage.configandvolume.config, but keeps them for compatibility, so we can merge to master branch.spans.idtospans.namefor clarityvolumes.avg_obj_sizeandvolumes.fragment_sizesupportIf this looks good, I'll work on implementation.