File tree Expand file tree Collapse file tree 2 files changed +13
-1
lines changed
Expand file tree Collapse file tree 2 files changed +13
-1
lines changed Original file line number Diff line number Diff line change @@ -43,7 +43,7 @@ Save to Hugging Face:
4343df.write.format(" huggingface" ).save(" username/my_dataset" )
4444# Or pass a token manually
4545df.write.format(" huggingface" ).option(" token" , " hf_xxx" ).save(" username/my_dataset" )
46- ```
46+ ```
4747
4848## Advanced
4949
@@ -91,3 +91,14 @@ huggingface datasource enabled for pyspark 3.x.x (backport from pyspark 4)
9191
9292The import is only necessary on Spark 3. x to enable the backport.
9393Spark 4 automatically imports `pyspark_huggingface` as soon as it is installed, and registers the " huggingface" data source.
94+
95+
96+ # # Development
97+
98+ [Install uv](https:// docs.astral.sh/ uv/ getting- started/ installation/ ) if not already done.
99+
100+ Then, from the project root directory, sync dependencies and run tests.
101+ ```
102+ uv sync
103+ uv run pytest
104+ ```
Original file line number Diff line number Diff line change @@ -5,6 +5,7 @@ description = "A DataSource for reading and writing HuggingFace Datasets in Spar
55authors = [
66 {name = " allisonwang-db" , email = " allison.wang@databricks.com" },
77 {name = " lhoestq" , email = " quentin@huggingface.co" },
8+ {name = " wengh" , email = " wenghy02@gmail.com" },
89]
910license = {text = " Apache License 2.0" }
1011readme = " README.md"
You can’t perform that action at this time.
0 commit comments