CS 471000 Introduction to Database Systems

Implementation, architectural design, and trade-offs.

Description

This course provides an overview of the current database management systems in the cloud, and explains how they are different from traditional database systems. The goal is to get students familiar with some well-known implementations like NoSQL databases, Google BigTable, Google MegaStore, and Google Spanner etc., and more importantly, to help students make better decisions on the design tradeoffs when configuring/building their own database systems given a particular set of target applications (tenants) in mind.

Proper understanding of Java/OOP programming and data structure is required.

Syllabus

Instructor

Teaching Assistants

Time & Location

  • Lecture: Mon. 15:30-17:20 at Delta 103
  • Lab: Thur. 14:20-15:10 at Delta 103
  • Office hour: Thur. 13:20-14:10 at Delta 723

Grading Policy

  • Assignments (x5): 50%
  • Quiz: 15%
  • Midterm exam: 15%
  • Final project : 20%

Prerequisits

This course is intended for senior undergraduate and junior graduate students who understand

  • OOP Programming,
  • Multi-threaded Programming,
  • Data Structure, and
  • Version Control.
We use Java as the main programming language throughout the course. Although not required, background knowledge about operating system will be helpful.

Announcement

  • 2021/6/11 - Lab9: slides announced (ARIES)

  • 2021/6/8 - Lecture13: slides announced

  • 2021/6/7 - Final Project: slides announced

  • 2021/5/31 - Lab12: slides announced

Curriculum

If you have any feedback, feel free to contact: shwu [AT] cs.nthu.edu.tw

Lecture 00

Introduction

What's Database? | About This Course... | FAQ

Slides

Lecture 01

Using a DBMS

Main Features of a DBMS | Data Models

Slides

Using PostgreSQL

Using PostgreSQL

Slides

Java Concurrency

Java Concurrency

Slides

Lecture 02

Data Modeling

ER & Relational Models | Weak Entities | Functional Dependencies | Normal Forms

Slides

General Rules For Assignments

The announcement the general rules for the assignments in this course and the introduction to the platform we use to manage assignments.

Slides

Introduction to Git

This lab guides you through the main idea of version control systems and the basic usage of Git.

Slides

Assignment 1

Given some scenario, design the ER model.

GitLab

Lecture 03

Architecture and Interfaces

Architecture Overview | SQL | JDBC | Native Interface | RecordFile | MetaData

Video Slides

Using VanillaDB

Basic tutorial of VanillaDB

Slides GitLab

Lecture 04

Server and Threads

Introduction | Threads Processes | Supporting Concurrent Clients | Embedded Clients vs. Remote Clients | RMI | JDBC Implementation

Video Slides

VanillaCore Walkthrough Part 1

This lab guides you through the VanillaCore. (Server, Remote Access, Utilities)

Slides

Introducing Benchmark Project

This lab guides you through the Benchmark Project.

Slides

Assignment 2

Implement JDBC version and Stored Procedures version of Read/Write transaction in VanillaDB.

GitLab

Lecture 05

Query Processing

Overview | Parsing & Verification | Parser & SQL Data | Predicates | Predicates | Scans | Plans | Assignment

Video Slides

VanillaCore Walkthrough Part 2

This lab guides you through the VanillaCore. (Query)

Slides

Assignment 1 Solution

Explain Assignment 1

Slides

Assignment 3

Implement Explain SQL operation in VanillaDB.

GitLab

Lecture 06

Data Access and File Management

Storage Engine | I/O Interfaces | Compromised File Management | Implementation: Page & FileMgr

Video Slides

VanillaCore Walkthrough Part 3

This lab guides you through the VanillaCore. (File Access)

Slides

Lecture 07

Memory Management

Buffer Pools & Pinning | Buffer Replacement | Pool Size & Deadlock Handling | ACID & Logging | Caching Logs

Video Slides

VanillaCore Walkthrough Part 4

This lab guides you through the VanillaCore. (Memory Management)

Slides

Assignment 4

Optimize file and buffer modules of VanillaCore.

Slides GitLab

Lecture 08

Record Management

Record Management | Design Considerations for Record Manager | The VanillaCore Record Manager

Video Slides

VanillaCore Walkthrough Part 5

This lab guides you through the VanillaCore. (Record)

Slides

Lecture 09

Transaction Management Part I : Concurrency Control

Overview of Transaction API | Schedules | Anomalies | 2PL & S2PL | Deadlock Handling | Assignment | Multi-Granularity Locking | Phantoms | Isolation Levels | Meta Structure | CC in VanillaCore

Video Slides

Transaction Management Part II : Recovery

Recap: WAL | Physical Logging & Tx Rollbacks | Undo Recovery | Undo-Redo Recovery | Repeated Failures | Checkpointing

Video Slides

ARIES

Overview of ARIES

Video Slides

VanillaCore Walkthrough Part 6

This lab guides you through the VanillaCore. (Lock)

Slides

Assignment 5

Implement two version two phase locking protocol.

GitLab Slides

Lecture 10

Cloud Database

Introducing SAE for cloud DBMS | Data Partitioning for Scalability | Replication for Availability | Case Study

Slides

NewSQL and Determinism

NewSQL | Determinism for SAE | Deterministic DDBMS | Performance Study

Slides

Assignment 4 Solution

Explain Assignment 4

Slides

Lecture 11

Group Communication

Reliable Broadcast | Consensus | Paxos | Zab

Video Slides

Lecture 12

Query Optimization

Cost Estimation | Histogram

Video Slides

Query Optimization Lab & Introduction of VanillaComm

Query Optimization Lab & Introduction of VanillaComm

Video Slides Slides

Lecture 13

Indexing

Hash Indexing | B-Tree Indexing | Index Locking

Video Slides

Final Project

Final Project Pre-announcement

Final project pre-announcement

Slides Assigned Readings

Final Project

Introduction of final project

Video Slides

Introduction to ElaSQL and ElaSQLBench

Introduction to ElaSQL and ElaSQLBench

Video Slides

Hermes Walkthrough

This lab guides you through the Hermes.

Video Slides

Final Project Presentation Announcement

Detail of final project presentation

Video Slides

Resources

Following provides links to some useful online resources. If this course starts your DB journey, don't stop here. Enroll yourself in advanced courses (shown below) to learn more.

Other Course Materials

For more course materials (such as assignments, score sheets, etc.) and online forum please refer to the GitLab.

GitLab

Documentation

Describes the API operations for the VanillaDB in detail.

Java Doc

Reference Books

  • Raghu Ramakrishnan et al., Database Management Systems, 3 Edition, McGraw-Hill, 2002, ISBN: 0072465638

  • Abraham Silberschatz et al, Database System Concepts, 6 Edition, McGraw-Hill, 2010, ISBN: 0073523321

  • Edward Sciore, Database Design and Implementation, Wiley, 2008, ISBN: 0471757160

  • M.Tamer Özsu, Principles of Distributed Database Systems, 3 Edition, Springer, 2011, ISBN: 1441988335

  • Rachid Guerraoui et al., Introduction to Reliable Distributed Programming, Springer, 2006, ISBN: 3540288457