What is DuckDB?

Anyone who works with data knows — everything is easy until your file grows too big. Small CSVs open fine in Excel, Pandas feels snappy, and everything just works. But once you cross the million-rows line, things start to fall apart. Pandas slows down, your RAM takes off like a jet engine, and suddenly a “quick data check” becomes a bottleneck.

That’s exactly where DuckDB steps in.

At first glance, DuckDB might look like just another SQLite clone, but it’s built for a very different purpose: data analytics. If SQLite is designed for OLTP (Online Transaction Processing), DuckDB is its counterpart for OLAP (Online Analytical Processing). In other words, if you’re working with large datasets, doing lots of reads, filters, groupings, and aggregations, DuckDB might be your new best friend.

So, what makes DuckDB special?

Columnar Storage Engine

Unlike traditional row-based databases, DuckDB stores data column by column. This makes analytical queries incredibly fast — it only reads the columns you need, not the entire table. It's cache-friendly and memory-efficient by design.

Embedded by Default

DuckDB doesn’t require a server. It runs inside your application, just like SQLite. No daemons, no ports, no client/server setup. You open it, run your queries, close it — that's it. Perfect for CLI tools, scripts, desktop apps, or embedded devices.

Fully SQL-Compliant

It speaks modern SQL fluently. DuckDB supports complex queries, joins, CTEs, window functions, GROUP BY, HAVING — the works. It also includes special functions like read_csv_auto() and read_parquet() for working with external data sources directly.

Native Support for Modern File Formats

DuckDB can read and write CSV, Parquet, and even JSON files natively — no conversion or imports needed. This is a game-changer for working with real-world data pipelines.

Language Flexibility

Whether you're writing Python scripts, Go backend services, or R notebooks, DuckDB has bindings for it. You get the same speed and simplicity across different environments.

Who is DuckDB for?

• Data scientists — to explore huge CSVs or Parquet files in-memory
• Backend developers — to add embedded reporting to apps without extra infrastructure
• ETL engineers — to pre-process data before pushing to cloud warehouses
• DevOps teams — to filter and summarize logs before shipping them to Elasticsearch or Loki

DuckDB is that sweet spot between heavy-duty OLAP systems (like ClickHouse or BigQuery) and lightweight databases (like SQLite). It’s modern, fast, embedded, and surprisingly versatile. Whether you're exploring a messy CSV, building an in-app dashboard, or filtering a Parquet file in production — DuckDB just works.

Using DuckDB with Golang

One of the best things about DuckDB is how easily it integrates with multiple programming languages — and Go is no exception. If you’re building a backend service in Go and want powerful, in-process analytics without spinning up a separate database or external engine, DuckDB might be your new favorite tool.

Let’s walk through how to set it up and use it.

1. Installation

Start by initializing your Go module and adding the DuckDB driver:

go mod init duckdb-demo
go get github.com/marcboeker/go-duckdb

DuckDB integrates with Go via the standard database/sql package — so if you’ve worked with SQLite or Postgres in Go, this will feel very familiar.

2. Basic Usage (In-Memory)

Here’s a quick example: we’ll open an in-memory DuckDB instance, create a table, insert data, and query it:

package main

import (
	"database/sql"
	"fmt"
	"log"

	_ "github.com/marcboeker/go-duckdb"
)

func main() {
	db, err := sql.Open("duckdb", "")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	_, err = db.Exec(`CREATE TABLE users (id INTEGER, name TEXT)`)
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec(`INSERT INTO users VALUES (1, 'Savaş'), (2, 'Atlas')`)
	if err != nil {
		log.Fatal(err)
	}

	rows, err := db.Query(`SELECT id, name FROM users`)
	if err != nil {
		log.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var id int
		var name string
		if err := rows.Scan(&id, &name); err != nil {
			log.Fatal(err)
		}
		fmt.Printf("User: %d - %s\n", id, name)
	}
}

3. Using a Persistent .db File

Instead of in-memory mode, you can persist data by passing a file name:

db, err := sql.Open("duckdb", "data.db")

This is especially handy for quick analysis on large datasets that would otherwise require Pandas or custom tooling.

5. Reading Parquet Files

Need to query analytics-friendly Parquet files? DuckDB makes it just as easy:

rows, err := db.Query(`SELECT * FROM read_parquet('logs.parquet')`)

With this, DuckDB becomes a mini-ETL engine — perfect for backend services or even local data crunching.

Why It Matters

DuckDB brings powerful OLAP-style querying directly into your Go application — no servers, no infrastructure, just simple, blazing-fast SQL. Whether you’re building a reporting tool, a local analytics dashboard, or a developer utility, DuckDB’s Go integration gives you speed, simplicity, and flexibility in one tiny package.