Skip to content

Commit edc0305

Browse files
CreateDataSourceTableCommand Logical Command
1 parent 3a87f8d commit edc0305

File tree

2 files changed

+34
-17
lines changed

2 files changed

+34
-17
lines changed

docs/configuration-properties.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -565,7 +565,9 @@ Default: `true`
565565

566566
Use [SQLConf.sortBeforeRepartition](SQLConf.md#sortBeforeRepartition) method to access the current value.
567567

568-
## <span id="spark.sql.hive.filesourcePartitionFileCacheSize"> hive.filesourcePartitionFileCacheSize
568+
## <span id="spark.sql.hive"> spark.sql.hive
569+
570+
### <span id="spark.sql.hive.filesourcePartitionFileCacheSize"> filesourcePartitionFileCacheSize
569571

570572
**spark.sql.hive.filesourcePartitionFileCacheSize**
571573

@@ -582,7 +584,7 @@ Used when:
582584

583585
* `FileStatusCache` is requested to [look up the system-wide FileStatusCache](files/FileStatusCache.md#getOrCreate)
584586

585-
## <span id="spark.sql.hive.manageFilesourcePartitions"> hive.manageFilesourcePartitions
587+
### <span id="spark.sql.hive.manageFilesourcePartitions"> manageFilesourcePartitions
586588

587589
**spark.sql.hive.manageFilesourcePartitions**
588590

docs/logical-operators/CreateDataSourceTableCommand.md

Lines changed: 30 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,32 +4,47 @@ title: CreateDataSourceTableCommand
44

55
# CreateDataSourceTableCommand Logical Command
66

7-
`CreateDataSourceTableCommand` is a RunnableCommand.md[logical command] that <<run, creates a new table>> (in a [SessionCatalog](../SessionCatalog.md)).
7+
`CreateDataSourceTableCommand` is a [LeafRunnableCommand](LeafRunnableCommand.md) that represents a [CreateTable](CreateTable.md) logical operator (with [DataSource Tables](../connectors/DDLUtils.md#isDatasourceTable)) at execution.
88

9-
`CreateDataSourceTableCommand` is created when [DataSourceAnalysis](../logical-analysis-rules/DataSourceAnalysis.md) posthoc logical resolution rule resolves a CreateTable.md[CreateTable] logical operator for a non-Hive table provider with no query.
9+
`CreateDataSourceTableCommand` uses the [SessionCatalog](../SessionState.md#catalog) to [create a table](../SessionCatalog.md#createTable) when [executed](#run).
1010

1111
## Creating Instance
1212

1313
`CreateDataSourceTableCommand` takes the following to be created:
1414

15-
* [[table]] [CatalogTable](../CatalogTable.md)
16-
* [[ignoreIfExists]] `ignoreIfExists` Flag
15+
* <span id="table"> [CatalogTable](../CatalogTable.md)
16+
* <span id="ignoreIfExists"> `ignoreIfExists` flag
1717

18-
=== [[run]] Executing Logical Command -- `run` Method
18+
`CreateDataSourceTableCommand` is created when:
1919

20-
[source, scala]
21-
----
22-
run(sparkSession: SparkSession): Seq[Row]
23-
----
20+
* [DataSourceAnalysis](../logical-analysis-rules/DataSourceAnalysis.md) posthoc logical resolution rule is executed (to resolve [CreateTable](CreateTable.md) logical operators with [DataSource Tables](../connectors/DDLUtils.md#isDatasourceTable))
2421

25-
NOTE: `run` is part of <<RunnableCommand.md#run, RunnableCommand Contract>> to execute (run) a logical command.
22+
## Executing Command { #run }
2623

27-
`run` requests a session-scoped `SessionCatalog` to [create a table](../SessionCatalog.md#createTable).
24+
??? note "RunnableCommand"
2825

29-
NOTE: `run` uses the input `SparkSession` to SparkSession.md#sessionState[access SessionState] that in turn is used to SessionState.md#catalog[access the current SessionCatalog].
26+
```scala
27+
run(
28+
sparkSession: SparkSession): Seq[Row]
29+
```
3030

31-
Internally, `run` [creates a BaseRelation](../DataSource.md#resolveRelation) to access the table's schema.
31+
`run` is part of the [RunnableCommand](RunnableCommand.md#run) abstraction.
3232

33-
CAUTION: FIXME
33+
`run` requests the [SessionCatalog](../SessionState.md#catalog) to [tableExists](../SessionCatalog.md#tableExists). With `ignoreIfExists` flag enabled, `run` exits. Otherwise, `run` reports a `TableAlreadyExistsException`.
3434

35-
NOTE: `run` accepts tables only (not views) with the provider defined.
35+
`run` uses the [locationUri](../CatalogStorageFormat.md#locationUri) as the `path` option.
36+
37+
`run` uses the [current database](../SessionCatalog.md#getCurrentDatabase) (of the [SessionCatalog](../SessionState.md#catalog)) unless defined.
38+
39+
`run` sets the [tracksPartitionsInCatalog](../CatalogTable.md#tracksPartitionsInCatalog) to the value of [spark.sql.hive.manageFilesourcePartitions](../configuration-properties.md#spark.sql.hive.manageFilesourcePartitions) configuration property.
40+
41+
`run` creates a [DataSource](../DataSource.md) to [resolveRelation](../DataSource.md#resolveRelation).
42+
43+
In the end, `run` requests the [SessionCatalog](../SessionState.md#catalog) to [create a table](../SessionCatalog.md#createTable).
44+
45+
??? note "AssertionError"
46+
47+
`run` expects the following (or reports an `AssertionError`):
48+
49+
* The [tableType](../CatalogTable.md#tableType) of the [CatalogTable](#table) is not `view`
50+
* The [provider](../CatalogTable.md#provider) of the [CatalogTable](#table) is defined

0 commit comments

Comments
 (0)