Directory
Published directory entries matching your search.
Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters. It provides high-level APIs in Java, Scala, Python and R, and supports Spark SQL, pandas API on Spark, MLlib, GraphX and Structured Streaming. The documentation covers downloads, the shell and client applications.
spark is a performance profiler for Minecraft clients, servers, and proxies, with a built-in profiler for diagnosing performance problems and bottlenecks. It also produces heap dumps, summarizes memory usage, monitors GC activity, and reports key metrics. The website serves as an online viewer for spark data.
Spark is a GitHub repository or organization with source code, releases, documentation, or project resources.
Spark is a fast and general engine for large-scale data processing.
Hydrosphere Mist is a serverless proxy for Spark clusters. It offers a functional programming framework and deployment model for Spark applications, with a REST HTTP and messaging API for Scala and Python Spark jobs. Sessions are managed across EMR, Hortonworks, Cloudera, DC/OS and vanilla Spark clusters.
Koalas, from Databricks, implements the pandas DataFrame API on top of Apache Spark. The repository is marked deprecated and in maintenance mode; Koalas supports Apache Spark 3.1 and below, and PySpark is recommended directly for Spark 3.2 and above. It is available under the Apache-2.0 license.
SparkChess is a site for playing chess online for free. Users can play against the computer or challenge friends in multiplayer, and the site also offers puzzles and lessons. A premium version removes the 2,000 points limit on scores.
SparkDEX V4 is a decentralized trading protocol for swapping tokens and accessing on-chain liquidity.
Sparked Paste helps clean, compare, sort, transform, count, or format text in the browser.
SparkLend is a DeFi protocol for on-chain finance, liquidity, yield, trading, or asset management.
SparkleShare helps upload, sync, or manage files across cloud storage, remote drives, or shared storage services.
SparkNotes provides educational videos, lessons, tutorials, or self-learning resources.
DataComPy is a package for comparing two DataFrames or tables. It supports Pandas, Polars, Spark (classic and Spark Connect), and Snowflake, and was originally created as a replacement for SAS PROC COMPARE for Pandas DataFrames. It prints stats and lets users adjust required match accuracy.
A listener that streams your spark events logs to delight, a free and improved spark UI.
A distributed machine learning framework Apache Spark.
A genomics processing engine and specialized file format built using Apache Avro, Apache Spark and Parquet. Apache 2 licensed.
Apache Zeppelin is a web-based notebook for data-driven, interactive data analytics and collaborative documents with SQL, Scala, Python, R and more. It uses interpreters to plug in language and data-processing backends, including built-in Apache Spark integration, and includes basic charts, pivot charts, dynamic forms and paragraph publishing to embed results in websites.
Delta Lake is an open-source storage framework designed to support the construction of a Lakehouse architecture. It is compatible with multiple compute engines including Apache Spark, PrestoDB, Flink, Trino, and Hive, as well as associated APIs.
dtreeviz is a Python library for decision tree visualization and model interpretation. It supports scikit-learn, XGBoost, Spark MLlib, LightGBM and TensorFlow, and uses an adaptor object obtained from dtreeviz.model() to extract model information for visualization.
A Clojure dataframe library that runs on Apache Spark.
H2O Sparkling Water is an open-source GitHub repository maintained by the h2oai organization. It provides H2O machine learning functionality that can run inside Apache Spark clusters. The repository includes getting started guides, user documentation, binary download options, and resources for testing the tool.
lakeFS is a source-available data version control tool that turns object storage into a Git-like repository, allowing data lakes to be managed like code. It supports AWS S3, Azure Blob Storage and Google Cloud Storage, is API compatible with S3, and works with frameworks such as Spark, Hive, AWS Athena, DuckDB and Presto.
Latest is a free and open-source macOS utility app that checks whether installed apps are up to date, shows which apps changed and what changed, and allows updating them. It supports apps from the Mac App Store and those using Sparkle for updates.
Luigi is a Python package for building complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures and command line integration, and includes Hadoop support. It is intended to stitch together long-running tasks such as Hive, Pig, Hadoop, Spark or database jobs.
