You can also interact with the SQL interface using the command-line SQL from within another programming language the results will be returned as a Dataset/DataFrame. For more on how toĬonfigure this feature, please refer to the Hive Tables section. Spark SQL can also be used to read data from an existing Hive installation. One use of Spark SQL is to execute SQL queries. The spark-shell, pyspark shell, or sparkR shell. This unification means that developers can easily switch back and forth betweenĭifferent APIs based on which provides the most natural way to express a given transformation.Īll of the examples on this page use sample data included in the Spark distribution and can be run in The same execution engine is used, independent of which API/language you are using to express theĬomputation. Interact with Spark SQL including SQL and the Dataset API. ![]() Spark SQL uses this extra information to perform extra optimizations. Unlike the basic Spark RDD API, the interfaces providedīy Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. Spark SQL is a Spark module for structured data processing.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |