-
-
Notifications
You must be signed in to change notification settings - Fork 262
[ENH] V1 → V2 API Migration - datasets #1608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
JATAYU000
wants to merge
283
commits into
openml:main
Choose a base branch
from
JATAYU000:dataset_resource
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,207
−689
Open
Changes from all commits
Commits
Show all changes
283 commits
Select commit
Hold shift + click to select a range
c955f43
Modify reviews, feature and qualities
JATAYU000 4c75e16
undo changes in tasks/functions.py
geetu040 5762185
Merge branch 'main' into migration
geetu040 021a1e1
made requested changes
satvshr 4c4a12c
Merge branch 'main' into issue1564
satvshr 1d91220
made requested changes
satvshr 3e26ace
Merge branch 'issue1564' of https://github.com/satvshr/openml-python …
satvshr 0060b2e
fixed bugs
satvshr 65ba66b
fixed bugs
satvshr 317c6e9
fixed bugs
satvshr 503ab82
fixed bugs
satvshr fd7ea2b
Merge branch 'main' into issue1564
satvshr fa3cd40
bug fixing
satvshr 3ad7268
Merge base pr
JATAYU000 3e7c415
Download methods
JATAYU000 8933cd8
Refactored functions
JATAYU000 dfa0ab7
Update todos, topic endpoints
JATAYU000 7e9bc1f
Merge branch 'main' into migration
geetu040 c603383
add tests directory
geetu040 ff6a8b0
use enum for delay method
geetu040 f01898f
implement cache
geetu040 5c4511e
refactor clients
geetu040 05dec9c
fixed minio methods and edit migration
JATAYU000 2012a72
Merge base pr
JATAYU000 35f450a
Updated Minio, download methods
JATAYU000 9d0098f
Merge branch 'main' into issue1564
fkiraly 5080752
checksum for arff download
JATAYU000 24582ef
use cache, new enpoints
JATAYU000 d789e83
lazy imports
JATAYU000 43276d2
fix import in resources/base.py
geetu040 1206f69
refactor and add exception handling
geetu040 bde5942
Merge branch 'main' into issue1564
satvshr 4948e99
refactor resources/base/
geetu040 a354167
implement delete
geetu040 1fe7e3e
implement publish and minor refactoring
geetu040 8d51fea
Fixes some tests
JATAYU000 bd85ec4
fix conflicts
JATAYU000 54a3151
implement tag/untag
geetu040 2b6fe65
implement fallback
geetu040 f4d6e75
code 103, dataset download
JATAYU000 0c16547
New tests partial
JATAYU000 fa53f8d
add test_http.py
geetu040 2b2db96
add uses_test_server marker
geetu040 f4c739d
Update delete test
JATAYU000 b87feb5
Merge commit 'refs/pull/1576/head' into dataset_resource
JATAYU000 c9617f9
implement reset_cache
geetu040 6bb20c3
Post update, v2 tests additions
JATAYU000 7d7fa43
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 5bc37b8
fixes with publish/delete
geetu040 08d9916
fix cache_key in tests
geetu040 8caba11
update _not_supported
geetu040 b4a817a
Merge commit '/pull/1576'
JATAYU000 1913c10
add 'get_api_config' skeleton method
SimonBlanke 7681949
remove 'APISettings'
SimonBlanke 01840a5
impl. 'get_api_config'
SimonBlanke 26ed4c1
add singleton pattern for settings
SimonBlanke c588d0c
add 'reset_settings'
SimonBlanke b6ff720
remove unused code
SimonBlanke 80d5afc
reimplement usage of v1 settings config
SimonBlanke f47112c
first try v2, fallback to v1 if not available
SimonBlanke d44cf3e
reimplement singelton without the use of 'global'
SimonBlanke ea7dda1
add explanations
SimonBlanke f0e5947
change usage of settings to new impl.
SimonBlanke edcd006
add explanations
SimonBlanke cde0aae
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] aa1e560
move to config: APIVersion, ResourceType
geetu040 06b8497
remove api_context entirely
geetu040 384da91
major refactor
geetu040 1878138
more refactoring with setup/
geetu040 dc26e01
implement APIBackend as controller
geetu040 e2d059b
move enums
geetu040 d156ad4
module level imports
geetu040 d7a3788
module level import for _backend
geetu040 b5b9ef6
module level import for tests
geetu040 68820fe
Merge branch 'main' into migration
geetu040 7288284
New tests,cache exception in download
JATAYU000 567eca4
add test: test_tag_and_untag
geetu040 d23790b
Merge, post fix
JATAYU000 95e8890
md5_checksum, and request fix
JATAYU000 16c9251
Merge branch 'openml:main' into dataset_resource
JATAYU000 23fe19b
Merge commit /pull/1576
JATAYU000 be29dc9
Merge branch 'dataset_resource'
JATAYU000 b2287c3
implement get/set_config_values
geetu040 b7e285e
improve APIBackend.set_config_values
geetu040 fd43c48
use LegacyConfig
geetu040 f4aab6b
Revert "use LegacyConfig"
geetu040 d43cf86
implement _sync_api_config
geetu040 3e323ed
update tests with _sync_api_config
geetu040 9195fa6
rename config: timeout -> timeout_seconds
geetu040 5342eec
use timedelta for default ttl value
geetu040 adc0e74
update tests, adds v2/fallback
geetu040 bfb2d3e
add MinIOClient in TestBase
geetu040 707e1f1
publish,tag methods need testing
JATAYU000 cabaecf
fix linting for builder
geetu040 79cf49c
new migration tests
JATAYU000 5c8791a
Merge /1576
JATAYU000 85c1113
fix unbound variables: "code", "message"
geetu040 39bf86a
use requests.Session()
geetu040 7b66677
remove "timeout_seconds" entirely
geetu040 d2224c4
update/refactor tests
geetu040 9608c36
remove unused current_api_version from TestAPIBase
geetu040 f6bc7f7
make TestAPIBase inherit TestBase
geetu040 baa3a38
nits: test classes
geetu040 29c93d1
Review changes, new tests
JATAYU000 7674b3a
Merge bse migration
JATAYU000 ddb0774
Doc strings
JATAYU000 52b93fe
minor fix in _sync_api_config
geetu040 ec9477f
chore: rerun CI
geetu040 cea6188
delete mock, decorator
JATAYU000 3d4e84d
Merge base
JATAYU000 839bd33
delete url in test
JATAYU000 8417349
New test design
JATAYU000 10d134a
remove duplicates in _api/resources/__init__.py
geetu040 935f0f4
implement HTTPClient.download and add tests
geetu040 9514df8
add docstrings
geetu040 09f9ad6
Review changes
JATAYU000 0b52427
Merge base pr
JATAYU000 53bee94
update minio
geetu040 33b4ca0
make delay functions static
geetu040 a6b9a45
rename: retry_raise_e -> exception
geetu040 f924b32
use context-manager for requests.Session
geetu040 541b0f2
remove "assert response is not None"
geetu040 acb173f
verify checksum before caching
geetu040 3e8d1f0
update tests
geetu040 f83bdb5
minor fix in ResourceV1API.untag
geetu040 969c7d8
Merge branch 'main' into dataset_resource
JATAYU000 2a42712
remove cache.ttl
geetu040 001caad
replace config.cache.dir with config.cache_dir
geetu040 fb38a2d
make HTTPClient.cache compulsory
geetu040 03c4ca9
remove unused OpenMLCacheRequiredError
geetu040 8d708fd
implement and use TestAPIBase._create_resource
geetu040 4f75bba
make ResourceAPI.minio compulsory
geetu040 164f66f
Merge branch 'main' into migration
geetu040 c4dae43
rename: use_cache -> enable_cache; reset_cache -> refresh_cache
geetu040 36c20a2
use server config from TestBase
geetu040 ab3c1eb
tests: mock HTTP post calls to prevent race conditions
geetu040 0fc3c74
Merge bse_migration into dataset_resource
JATAYU000 741a66b
rename cache params
JATAYU000 81dff8d
Merge branch 'dataset_resource'
JATAYU000 27ac86f
Minio assertions, other reviews
JATAYU000 2a488ca
Merge branch 'main' into migration
geetu040 cbc7194
Merge base migration
JATAYU000 599c7e1
remove hardcoded server in TestHTTPClient.test_cache
geetu040 2867862
fix docstring in _resolve_default_cache_dir
geetu040 f09f3cd
fix docstring in ResourceAPI
geetu040 5f731ce
remove duplicates in __all__
geetu040 bad7842
remove ttl related code/docs
geetu040 aefdb38
remove delay methods in HTTPClient
geetu040 0f40b02
minor fix in _resolve_default_cache_dir
geetu040 7ac1672
update FallbackProxy
geetu040 6ac1dfe
simplify _backend creation
geetu040 62924c9
Merge branch 'main' into migration
geetu040 27696bb
req changes
satvshr 190face
resolve conflicts
satvshr 95daaa6
remove old config file
satvshr 7841ea8
added OPENML_TEST_SERVER_ADMIN_KEY_ENV_VAR
satvshr cc515aa
bug fixing
satvshr e6a92df
armagh fix
satvshr 1b8c22a
update content_type check
geetu040 fc839a6
Revert "make delay functions static"
geetu040 1c922af
Revert "remove delay methods in HTTPClient"
geetu040 ffa9ce9
Merge branch 'main' into migration
geetu040 a7b2d21
allow api_key=None
geetu040 27fe790
add tests for api_key=None
geetu040 8965112
update cache not found message
geetu040 72ea1a4
update docs for path in HTTPCache
geetu040 a696c49
remove elapsed from cached meta
geetu040 755636d
move self.headers to _HEADERS
geetu040 d07af34
fix indentation in docstrings of _resolve_default_cache_dir
geetu040 2d9c8ec
Update openml/_api/clients/http.py
geetu040 002b989
Merge branch 'main' into migration
geetu040 045d896
move _handle_delete_exception and_get_endpoint_name, legal_resources
geetu040 c437966
set HTTPClient.headers
geetu040 e27470a
remove main_tag
geetu040 d04d956
remove and merge TestAPIBase into TestBase
geetu040 9263f7f
minor change in TestHTTPClient.test_cache
geetu040 79dea29
make HTTPClient.request private
geetu040 f6497c2
Revert "update FallbackProxy"
geetu040 dce7f54
use st_ctime instead of st_ctime for cache refresh test
geetu040 40dd460
Merge branch 'main' into issue1564
geetu040 0fc917c
majore config refactor
geetu040 3d86b18
Merge branch 'pr-1577' into migration
geetu040 aba3d3e
update _config.py
geetu040 d99d54d
update test_openml_cache_dir_env_var
geetu040 dc22e3a
fix mutable SERVERS_REGISTRY
geetu040 7318573
update set_api_version for fallback
geetu040 29ef187
minor fix
geetu040 cf94c89
fixes for test_config
geetu040 298fbda
fixes in conftest urls
geetu040 9870502
update test_http.py
geetu040 33065c2
undo changes with test_openml_cache_dir_env_var
geetu040 76b92bb
fix server mode in test_config.py
geetu040 419edcb
move _HEADERS to confing
geetu040 cb6d937
add fixtures for migration tests
geetu040 8544c8a
update test_http.py with fixtures
geetu040 d4c413b
update test_versions.py
geetu040 fab1a15
update test_versions.py
geetu040 6392be8
Merge base-migration
JATAYU000 276324a
fix error message in HTTPClient.server
geetu040 73f7594
fixes in test_versions.py: use DummyTaskAPI instead of TaskAPI
geetu040 2ee7fa3
add clients in openml._backend
geetu040 4f37607
skip parquet env var
JATAYU000 c74754a
Merge base-migration
JATAYU000 2473208
Updated test,admin fixture
JATAYU000 7afb0e3
code qulity Reviews
JATAYU000 3b96559
Test fixes
JATAYU000 ea80785
remove unnecessary
JATAYU000 83a2e80
Fix mock delete
JATAYU000 9eb6c90
Exception review
JATAYU000 4be5bbd
fixes with openml.config.[server|apikey] leakage
geetu040 9027c01
remove unused fixtures: use_api_[v1|v2]
geetu040 c1efdeb
Merge base-mgration
JATAYU000 dd048d5
mock requests
JATAYU000 98041ed
skip v2 test for now
JATAYU000 e5461a9
add more config tests
geetu040 7d899a9
make SERVERS_REGISTRY private
geetu040 8587414
fix marker: uses_test_server->test_server
geetu040 23a3450
fix UserWarning
geetu040 ac28f82
update fixture: with_server
geetu040 4a66245
req changes
satvshr c762fb4
Merge branch 'issue1564' of https://github.com/satvshr/openml-python …
satvshr 77c21f2
Update openml/_api/clients/http.py
geetu040 eac24fc
Update tests/test_api/test_http.py
geetu040 2ed65fe
update test_get_uses_cached_response
geetu040 f3b07de
test_get_with_api_key
geetu040 29db3f1
use .arff instead of .bin in tests
geetu040 3b4e538
update test_download_creates_file to use md5_checksum
geetu040 8ac886b
update test_download_is_cached_on_disk
geetu040 305f4f0
update APIBackendBuilder
geetu040 b2bf164
Merge branch 'main' into migration
geetu040 e97e6c2
Update openml/_api/clients/http.py
geetu040 c66d73c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] aa54e8e
pre-commit fixes
geetu040 2d452d3
Merge branch 'main' into issue1564
geetu040 c235812
Merge branch 'main' into issue1564
fkiraly 39eb823
Trigger CI
satvshr 50eed37
Merge branch 'main' into migration
geetu040 7a000eb
Merge branch 'main' into issue1564
geetu040 79f6187
Merge branch 'main' into issue1564
geetu040 b1a9e7f
Merge branch 'pr-1577' into migration (merge conflicts)
geetu040 d716ecf
update server methods in config
geetu040 3c29e71
fix api-version leakage in tests
geetu040 b4ff0b2
remove unused migration code
geetu040 93155ee
debug ci: separate cache for each test-case
geetu040 d3cc9a7
update port for localhost
geetu040 a6b82f4
Revert "debug ci: separate cache for each test-case"
geetu040 3419973
rerun CI
geetu040 8de99b7
Merge branch 'main' into migration
geetu040 d0202b0
Merge base migration
JATAYU000 0fa9e3b
Fix tests for new test setup
JATAYU000 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| from .clients import ( | ||
| HTTPCache, | ||
| HTTPClient, | ||
| MinIOClient, | ||
| ) | ||
| from .resources import ( | ||
| API_REGISTRY, | ||
| DatasetAPI, | ||
| DatasetV1API, | ||
| DatasetV2API, | ||
| EstimationProcedureAPI, | ||
| EstimationProcedureV1API, | ||
| EstimationProcedureV2API, | ||
| EvaluationAPI, | ||
| EvaluationMeasureAPI, | ||
| EvaluationMeasureV1API, | ||
| EvaluationMeasureV2API, | ||
| EvaluationV1API, | ||
| EvaluationV2API, | ||
| FallbackProxy, | ||
| FlowAPI, | ||
| FlowV1API, | ||
| FlowV2API, | ||
| ResourceAPI, | ||
| ResourceV1API, | ||
| ResourceV2API, | ||
| RunAPI, | ||
| RunV1API, | ||
| RunV2API, | ||
| SetupAPI, | ||
| SetupV1API, | ||
| SetupV2API, | ||
| StudyAPI, | ||
| StudyV1API, | ||
| StudyV2API, | ||
| TaskAPI, | ||
| TaskV1API, | ||
| TaskV2API, | ||
| ) | ||
| from .setup import ( | ||
| APIBackend, | ||
| APIBackendBuilder, | ||
| _backend, | ||
| ) | ||
|
|
||
| __all__ = [ | ||
| "API_REGISTRY", | ||
| "APIBackend", | ||
| "APIBackendBuilder", | ||
| "DatasetAPI", | ||
| "DatasetV1API", | ||
| "DatasetV2API", | ||
| "EstimationProcedureAPI", | ||
| "EstimationProcedureV1API", | ||
| "EstimationProcedureV2API", | ||
| "EvaluationAPI", | ||
| "EvaluationMeasureAPI", | ||
| "EvaluationMeasureV1API", | ||
| "EvaluationMeasureV2API", | ||
| "EvaluationV1API", | ||
| "EvaluationV2API", | ||
| "FallbackProxy", | ||
| "FlowAPI", | ||
| "FlowV1API", | ||
| "FlowV2API", | ||
| "HTTPCache", | ||
| "HTTPClient", | ||
| "MinIOClient", | ||
| "ResourceAPI", | ||
| "ResourceV1API", | ||
| "ResourceV2API", | ||
| "RunAPI", | ||
| "RunV1API", | ||
| "RunV2API", | ||
| "SetupAPI", | ||
| "SetupV1API", | ||
| "SetupV2API", | ||
| "StudyAPI", | ||
| "StudyV1API", | ||
| "StudyV2API", | ||
| "TaskAPI", | ||
| "TaskV1API", | ||
| "TaskV2API", | ||
| "_backend", | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| from .http import HTTPCache, HTTPClient | ||
| from .minio import MinIOClient | ||
|
|
||
| __all__ = [ | ||
| "HTTPCache", | ||
| "HTTPClient", | ||
| "MinIOClient", | ||
| ] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.