Please let me begin this article with a quote of Bruce Lee, one of my favorites martial artists:
Before I studied the art, a punch to me was just like a punch, a kick just like a kick. After I learned the art, a punch was no longer a punch, a kick no longer a kick. Now that I’ve understood the art, a punch is just like a punch, a kick just like a kick. The height of cultivation is really nothing special. It is merely simplicity; the ability to express the outmost with the minimum. It is the halfway cultivation that leads to ornamentation.
In not a very different manner, that happened to me while using sqlite databases on Android, either using ORM libraries, like ORMLite, DBFlow and so on, or using alternative databases like Realm. A database, it became just that, a database, no abstraction layers, just a database.
Nowadays, GitHub is full of libraries for android that help managing an sqlite database. Almost all of them try to follow the ORM technique. But, like Bruce Lee, i went back to basics, and none of them gave me the freedom to use it like i want and have full control over queries and operations. In lot of projects, a database is just that, a simple database, so tr to not overload your project with libraries for just caching some data or storing simple models.
But working with vanilla SQLiteOpenHelper is kind of tedious, so I kept searching for the perfect database library till i found SqlDelight by Square. I can’t say it fulfills all my need but it helps me a lot.
DISCLAIMER: At the time of writing this article, the latest version of SQLDelight was 0.3.0 (2016-04-26).
As they say in the repositories README file:
SQLDelight generates Java models from your SQL CREATE TABLE statements. These models give you a type safe API to read & write the rows of your tables. It helps you to keep your SQL statements together, organized, and easy to access from Java.
Why SQLDelight:
- All your SQL statements are stored in
.sqfiles. Yo can easily versionate all your database changes into your CVS - Generates schema models that help creating and querying the tables
- It gives you all the freedom of using vanilla SQLite but helping you with boilerplate code
- Helps mapping from android
Cursorto your custom model - Supports same types as
CursorandContentValuesbut adds support for your own custom types or evenENUM
Getting started with SQLDelight
First of all add the dependency to your build.gradle in the buildscript method. Latest versions can be found at Square’s sonatype:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.squareup.sqldelight:gradle-plugin:0.2.2'
}
}
Above step is enough to start using SQLDelight, but the guys at Square also made a IntelliJ plugin that gives you highlight and syntax support; I recommend installing it, can really help you avoiding some syntax errors.
Open Android Studio -> Preferences -> Plugins -> Search and install SQLDelight.
For this article i made a basic project to help as a proof of concept, you can clone it from GitHub.
Building database model
My sample project has com.alexsimo.delightfulpersistence as root package, that is important because at root of src/main you have to create a folder called sqlidelight and replicate your root package structure. After that, you end with following directories structure:
├── java
│ └── com
│ └── alexsimo
│ └── delightfulpersistence
│ ├── DelightfulApplication.java
│ └── database
│ ├── DatabaseManager.java
│ ├── DelightfulOpenHelper.java
│ ├── adapter
│ │ └── DateAdapter.java
│ └── model
│ ├── Author.java
│ └── Book.java
└── sqldelight
└── com
└── alexsimo
└── delightfulpersistence
└── database
└── model
├── Author.sq
├── Book.sq
└── BookAuthor.sq
In the model directory, bellow sqldelight tree, is where you will save your .sq files containing required SQL sentences for your application.
You can also observe more files, that i will later explain. Now let’s focus on creating our .sq files for the model. To keep the article short, i’ll only post the SQL sentences for creating and querying the Author and Book tables, but all of them can be found on the GitHub repository.
Author:
CREATE TABLE author (
_id LONG PRIMARY KEY AUTOINCREMENT,
name STRING NOT NULL,
birth_year CLASS('java.util.Calendar')
);
select_all:
select *
from author;
select_by_name:
select *
from author
where author.name = ?;
Book:
CREATE TABLE book (
_id LONG NOT NULL PRIMARY KEY AUTOINCREMENT,
isbn STRING NOT NULL,
title STRING NOT NULL,
release_year CLASS('java.util.Calendar')
);
select_all:
select *
from book;
select_by_title:
select *
from book
where book.title = ?;
select_by_isbn:
select *
from book
where book.isbn = ?;
The good thing about SQLDelight is that you also store your queries on .sq files and will help you replace the bindable parameters as where book.isbn = ?.
Creating database model
Now we’re ready to create our tables in the sqlite database, just build your project and SQLDelight should have generated BookModel and AuthorModel.
If we look inside BookModel we can see that it built the CREATE TABLE SQL statement, and the field properties representing each one of our table columns:
public interface BookModel {
String TABLE_NAME = "book";
String _ID = "_id";
String ISBN = "isbn";
String TITLE = "title";
String RELEASE_YEAR = "release_year";
String CREATE_TABLE = ""
+ "CREATE TABLE book (\n"
+ " _id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\n"
+ " isbn TEXT NOT NULL,\n"
+ " title TEXT NOT NULL,\n"
+ " release_year BLOB\n"
+ ")";
// continue bellow
Apart from the create table sentences, also generated the queries we stored in our .sq files.
String SELECT_ALL = ""
+ "select *\n"
+ "from book";
String SELECT_BY_TITLE = ""
+ "select *\n"
+ "from book\n"
+ "where book.title = ?";
String SELECT_BY_ISBN = ""
+ "select *\n"
+ "from book\n"
+ "where book.isbn = ?";
Note:In fact it generates more stuff, some lines bellow, you can see what i’m talking about.
With all this SQL sentences, we can easily create our tables, extending the SQLiteOpenHelper:
public class DelightfulOpenHelper extends SQLiteOpenHelper {
public static final String DB_NAME = "delightful.db";
public static final int DB_VERSION = 1;
private static DelightfulOpenHelper instance;
public static DelightfulOpenHelper getInstance(Context context) {
if (null == instance) {
instance = new DelightfulOpenHelper(context);
}
return instance;
}
private DelightfulOpenHelper(Context context) {
super(context, DB_NAME, null, DB_VERSION);
}
@Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(BookModel.CREATE_TABLE);
db.execSQL(AuthorModel.CREATE_TABLE);
db.execSQL(BookAuthorModel.CREATE_TABLE);
populate(db);
}
private void populate(SQLiteDatabase db) {
AuthorPopulator.populate(db);
BookPopulator.populate(db);
}
}
If you use the basic implementation of SQLiteOpenHelper that the official Android documentation says, you can have some concurrency problems, as show in the Dmytro’s blog post, or you if are spanish reader, in this article.
To handle the problem mentioned above, i created a wrapper for the SQLiteHelper:
DatabaseManager.java:
public class DatabaseManager {
private static AtomicInteger openCount = new AtomicInteger();
private static DatabaseManager instance;
private static DelightfulOpenHelper openHelper;
private static SQLiteDatabase database;
public static synchronized DatabaseManager getInstance() {
if (null == instance) {
throw new IllegalStateException(DatabaseManager.class.getSimpleName()
+ " is not initialized, call initialize(..) method first.");
}
return instance;
}
public static synchronized void initialize(DelightfulOpenHelper helper) {
if (null == instance) {
instance = new DatabaseManager();
}
openHelper = helper;
}
public synchronized SQLiteDatabase openDatabase() {
if (openCount.incrementAndGet() == 1) {
database = openHelper.getWritableDatabase();
}
return database;
}
public synchronized void closeDatabase() {
if (openCount.decrementAndGet() == 0) {
database.close();
}
}
}
The above class, I instantiate it on the Application class or using Dagger.
With a very simple android connected test (requires device or emulator), we can check if our tables were created successfully:
public class DatabaseShould extends CustomRunner {
@Override
@Before
public void setUp() throws Exception {
super.setUp();
DbCommon.deleteDatabase(context);
}
@Test
public void be_able_to_open_writable_database() throws Exception {
SQLiteDatabase db = givenWritableDatabase();
assertTrue(db.isOpen());
assertTrue(!db.isReadOnly());
}
@Test
public void have_created_tables() throws Exception {
SQLiteDatabase db = givenWritableDatabase();
HashSet<String> tables = givenAllTables();
Cursor cursor = db.rawQuery("SELECT name FROM sqlite_master WHERE type='table'", null);
cursor.moveToFirst();
do {
String table = cursor.getString(0);
tables.remove(table);
} while (cursor.moveToNext());
cursor.close();
assertTrue(tables.isEmpty());
}
private SQLiteDatabase givenWritableDatabase() {
return DbCommon.givenWritableDatabase(context);
}
private HashSet<String> givenAllTables() {
HashSet<String> tables = new HashSet<>();
tables.add(BookModel.TABLE_NAME);
tables.add(AuthorModel.TABLE_NAME);
tables.add(BookAuthorModel.TABLE_NAME);
return tables;
}
}
Also you can test the table’s column creation process as above, not very clean manner but i will be glad to hear from you how to improve the tests:
public class AuthorTableShould extends CustomRunner {
@Before
public void setUp() throws Exception {
super.setUp();
DbCommon.deleteDatabase(context);
}
@Test
public void have_created_all_columns() throws Exception {
HashSet<String> columns = givenAuthorColumns();
SQLiteDatabase db = givenWritableDatabase();
Cursor cursor = db.rawQuery("PRAGMA table_info(" + AuthorModel.TABLE_NAME + ")", null);
cursor.moveToFirst();
int columnNameIndex = cursor.getColumnIndex("name");
do {
String columnName = cursor.getString(columnNameIndex);
columns.remove(columnName);
} while (cursor.moveToNext());
assertTrue(columns.isEmpty());
cursor.close();
db.close();
}
private SQLiteDatabase givenWritableDatabase() {
return DbCommon.givenWritableDatabase(context);
}
private HashSet<String> givenAuthorColumns() {
HashSet<String> columns = new HashSet<>();
columns.add(AuthorModel._ID);
columns.add(AuthorModel.NAME);
columns.add(AuthorModel.BIRTH_YEAR);
return columns;
}
}
Inserting data with SQLDelight
Remember that in our CREATE TABLE definition in the .sq files, we used a custom type, java.util.Calendar. As is not a native SQLite type, we must create a adapter for it.
So go and create it, in my sample project, i called it DateAdapter:
public class DateAdapter implements ColumnAdapter<Calendar> {
@Override
public Calendar map(Cursor cursor, int columnIndex) {
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(cursor.getLong(columnIndex));
return calendar;
}
@Override
public void marshal(ContentValues values, String key, Calendar value) {
values.put(key, value.getTimeInMillis());
}
}
This should be enough, but we must tell SQLDelight how to use it, so go and implement the BookModel and AuthorModel generated by SQLDelight and override the inner Author and Book Marshall class and pass the calendar adapter.
@AutoValue public abstract class Author implements AuthorModel {
private final static DateAdapter DATE_ADAPTER = new DateAdapter();
public final static Mapper<Author> MAPPER =
new Mapper<>((Mapper.Creator<Author>) AutoValue_Author::new, DATE_ADAPTER);
public static final class Marshal extends AuthorMarshal<Marshal> {
public Marshal() {
super(DATE_ADAPTER);
}
}
}
By the moment ignore the MAPPER field, it will be explained when requesting data from the table. Observe the final class Marshal, there is where you tell SQLDelight it should use your adapter for the java.util.Calendar type. In the sample i use Google’s @AutoValue and RetroLambda to avoid some boilerplate code.
For inserting default data, i just created a dummy class called Book and Author Popullator:
public class AuthorPopullator {
public static void populate(SQLiteDatabase db) {
db.insert(AuthorModel.TABLE_NAME, null, new Author.Marshal().name("J. K. Rowling")
.birth_year(new GregorianCalendar(1965, 7, 31))
.asContentValues());
db.insert(AuthorModel.TABLE_NAME, null, new Author.Marshal().name("Bella Forests")
.birth_year(new GregorianCalendar(197, 17, 31))
.asContentValues());
db.insert(AuthorModel.TABLE_NAME, null, new Author.Marshal().name("Norah Roberts")
.birth_year(new GregorianCalendar(1950, 10, 10))
.asContentValues());
db.insert(AuthorModel.TABLE_NAME, null, new Author.Marshal().name("David Baldacci")
.birth_year(new GregorianCalendar(1960, 8, 5))
.asContentValues());
db.insert(AuthorModel.TABLE_NAME, null, new Author.Marshal().name("Jeff Wheeler")
.birth_year(new GregorianCalendar(1955, 13, 31))
.asContentValues());
}
}
As you can see, SQLDelight created for us a nice fluent builder, resulting in a more cleaner code.
Deleting data as you can expect follows the same process, you define the delete sql sentences in the .sq file and execute it using a SQLiteDatabase object.
Querying data with SQLDelight
In order to demonstrate how to query data using some sugar from SQLDelight, as above, i will use some tests:
@Test
public void be_able_to_return_cursor_with_all_default_authors() throws Exception {
SQLiteDatabase db = givenWritableDatabase();
Cursor cursor = db.rawQuery(AuthorModel.SELECT_ALL, new String[0]);
int AUTHOR_COUNT = 5;
assertTrue(cursor.getCount() == AUTHOR_COUNT);
}
@Test
public void map_cursor_with_domain_model() throws Exception {
SQLiteDatabase db = givenWritableDatabase();
Cursor cursor = db.rawQuery(AuthorModel.SELECT_ALL, new String[0]);
cursor.moveToFirst();
Author author = Author.MAPPER.map(cursor);
assertNotNull(author);
}
Take a closer look at Author.MAPPER.map(cursor). Yep! You’re right! It mapped our cursor to the model class. For me that’s a very nice feature, as it saves me some boilerplate mapping code, also is less prone to mistakes.
Migrations with SQLDelight
Actually there isn’t a official way of handling migrations, but there’s an open issue you can check for progress.
In my sample project, i used a dirty way to store migration sentences into .sq files.
I store the migration files in the same folder as model creation and queries files:
└── sqldelight
└── com
└── alexsimo
└── delightfulpersistence
└── database
└── model
├── Author.sq
├── Book.sq
├── BookAuthor.sq
├── Migration_v1.sq
└── Migration_v2.sq
If you look inside of Migration_v1.sq you can see that the SQLDelight IntelliJ plugin and compiler obligates you to start the .sq file with a CREATE TABLE statement, so i’m just adding a CREATE TABLE sentence with a dummy table i won’t create:
CREATE TABLE dummy(_id LONG NOT NULL PRIMARY KEY AUTOINCREMENT);
migrate_author:
ALTER TABLE author
ADD COLUMN country STRING NOT NULL;
Later in our custom SQLiteOpenHelper class we can use the migrate sentences in the onUpgrade() method:
@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
if (oldVersion < 2) {
db.execSQL(Migration_v1Model.MIGRATE_AUTHOR);
}
if (oldVersion < 3) {
db.execSQL(Migration_v2Model.MIGRATE_BOOK);
}
}
This isn’t the cleanest form of handling migrations but i’m sure the guys from Square will find better approach in future releases of SQLDelight.