Independent Science + Technology

Category: spark

🔥 Day 2: Understanding Spark Architecture – How Spark Executes Your Code Internally

Post date December 2, 2025
Post author By Sandeep
Post categories In bigdata, dataengineering, python, spark

Data Quality on Spark, Part 2: Soda

Post date November 25, 2025
Post author By Ivan Kurchenko
Post categories In data-engineering, data-quality, python, soda, spark

Spark Memory Explained: Executor, Driver, and Overhead Memory Demystified (with Real Examples)

Post date October 20, 2025
Post author By Pradosh Kumar
Post categories In big-data, data-analysis, data-science, programming, spark

A Beginner’s Guide to Big Data Analytics with Apache Spark and PySpark

Post date September 28, 2025
Post author By Amos Augo
Post categories In dataengineering, pyspark, spark

Turbocharging AI Sentiment Analysis: How We Hit 50K RPS with GPU Micro-services

Post date March 7, 2025
Post author By Vineeth Reddy Vatti
Post categories In cloud-storage, etl, GPU, kafka, kubernetes, microservices, sentiment-analysis, spark

Tiny URL Design

Post date March 3, 2025
Post author By Abhishek Prajapati
Post categories In kafka, mysql, redis, spark

Automatizando a Qualidade de Dados com DQX: Performance e praticidade

Post date February 27, 2025
Post author By Airton Lira junior
Post categories In databricks, dqx, python, spark

Superheroes of Spark

Post date February 18, 2025
Post author By Margaret O'Brien
Post categories In data-engineering, data-science, programming, python, spark

AWS Glue vs AWS Lambda: Comparativa Serverless para Ingeniería de Datos en AWS

Post date February 16, 2025
Post author By Jose Luis Ariza
Post categories In awsglue, lambda, serverless, spark

Big Boost for Flink & Spark SQL: Both Tools Just Got Updated!

Post date February 8, 2025
Post author By DogeKing
Post categories In flinksql, spark, sparksql, sql

Entendendo e aplicando estratégias de tunning Apache Spark

Post date November 7, 2024
Post author By Airton Lira junior
Post categories In databricks, pyspark, python, spark

[API Databricks como serviço interno] dbutils — notebook.run, widgets.getArgument, widgets.text e notebook_params

Post date November 2, 2024
Post author By Airton Lira junior
Post categories In databricls, jupyter, pyspark, spark

Análise de dados de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka

Post date October 28, 2024
Post author By Geazi Anc
Post categories In braziliandevs, dataengineering, python, spark

Leveraging PySpark.Pandas for Efficient Data Pipelines

Post date July 4, 2024
Post author By Felipe de Godoy
Post categories In dataengineering, pandas, python, spark

Comprehensive Guide to Schema Inference with MongoDB Spark Connector in PySpark

Post date June 27, 2024
Post author By Chetan Gupta
Post categories In bigdata, mongodb, pyspark, spark

Polars VS PySpark: Lazy Evaluation and Big Data

Post date May 12, 2023
Post author By Luís Oliveira
Post categories In data-engineering, data-science, polar, python, spark

Integrate Apache Spark and QuestDB for Time-Series Analytics

Post date April 6, 2023
Post author By Imre Aranyosi
Post categories In database, questdb, spark, tutorial

Explaining Distributed Systems Like I’m 5

Post date March 30, 2023
Post author By Sabrina
Post categories In beginners, devops, programming, spark

Text Clustering using Python and Spark

Post date March 29, 2023
Post author By Davide Gazzè - Ph.D.
Post categories In clustering, spark

From Hadoop to Spark: An In-Depth Look at Distributed Computing Frameworks

Post date January 30, 2023
Post author By Saeed Mohajeryami, PhD
Post categories In batch-processing, big-data, distributed-computing, spark, streaming

PySpark: A brief analysis to the most common words in Dracula, by Bram Stoker

Post date January 11, 2023
Post author By Geazi Anc
Post categories In dataengineering, datascience, python, spark

Why we don’t use Spark

Post date September 7, 2022
Post author By Karel Vanden Bussche
Post categories In bigdata, googlecloud, python, spark

Deep Dive into Apache Iceberg via Apache Zeppelin

Post date July 18, 2022
Post author By Jeff Zhang
Post categories In apacheiceberg, apachezeppelin, spark

4 Tips To Integrate Pytest With Pyspark Applications

Post date July 5, 2022
Post author By Pathairush Seeda
Post categories In data-analysis, python, software engineering, spark, testing

Run unpublished Spark notebooks in Azure Synapse

Post date May 12, 2022
Post author By José Fernando Costa
Post categories In Azure, data-engineering, pyspark, python, spark

Build a rest service from the command line, as simple as “every request has a response.”

Post date March 28, 2022
Post author By Thinking out code
Post categories In java, pingpong, restservice, spark

Data Lake explained

Post date January 11, 2022
Post author By Barbara
Post categories In Analytics, beginners, bigdata, spark

Run Spark locally with Docker

Post date January 7, 2022
Post author By Barbara
Post categories In beginners, docker, jupyter, spark

Spark is lit once again

Post date October 29, 2021
Post author By Mindaugas
Post categories In hacktoberfest, kubernetes, opensource, spark

Ultimate Guide to Data Engineer Interviews in 2021

Post date August 17, 2021
Post author By Nitesh Chaudhry
Post categories In big-data, data-engineering, interview, spark, sql

How to Authenticate Kafka Using Kerberos (SASL), Spark, and Jupyter Notebook

Post date July 19, 2021
Post author By Artem Gogin
Post categories In apache-spark, jupyter-notebook, kafka, kerberos, programming, pyspark, spark, spark-streaming

5 Best Big Data Frameworks You Can Learn in 2021

Post date June 19, 2021
Post author By javinpaul
Post categories In bigdata, java, programming, spark

Apache Spark Ecosystem

Post date April 30, 2021
Post author By Anello
Post categories In apache, apache-spark, big-data, data, spark

Nothing left to load.