Intro
How you store your data changes over time. Some cases, you need to modify current data to be fit in a new data model structure. If your using SQL DB, you can write a migration script and run it. But if you’re using Firebase Firestore, this task can be tricky since itself has no concept of migration.
In order to do that, you can use Firebase Cloud Function, write some code using Firebase Admin SDK, and run it. It’s possible, but what if you’re using free plan(you can’t use Cloud Function unless you give them a payment information) or using Dart?(according to pub.dev, you can’t use official firebase package in pure dart, you must use Flutter)
I looked it up on this topic and found an awesome package called fireway. It inspired me to work on a same problem, but in this case, with Pure Dart.
In this post, I would like to share what I’ve built and how to use it.
What I’ve used
- Dart console application
- google api related packages
-
service-account.jsonfor authentication -
dart_consolefor display text in a console -
versionfor dealing with migration version
Github repository for source code
You can clone this public repository and modify freely.
How to use this Dart application
(Assum you already cloned the repository)
1. Create service account json file
First, you need service account json file. It is needed in order to automate authentication(without login all the time). You can create service account file in Google Developer Console. The link below explains how to do it.
The content is something like this.
{
"type": "service_account",
"project_id": "PROJECT_ID",
"private_key_id": "KEY_ID",
"private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n",
"client_email": "SERVICE_ACCOUNT_EMAIL",
"client_id": "CLIENT_ID",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL"
}
WARNING → DO NOT upload this file to github publically! Anyone who has this file can lookup your project and do dirty thing. Add it to .gitignore!
You may want to move the service account file to your project root.
2. flutter pub get
Of course.
flutter pub get
3. Write your migration code
Under the lib/migrations/versions , you’ll find 0_0_1, 0_0_2, 0_0_3 dart files. You must follow semver versioning rules. But in the file name, use “_” instead of “.”
In 0_0_1.dart , there’s a same code.
import 'package:dart_console/dart_console.dart';
import 'package:firebaseapis/firestore/v1.dart';
import 'package:firestore_migration/config/config.dart';
import 'package:firestore_migration/src/crud_document.dart';
import 'package:firestore_migration/src/crud_field.dart';
import 'package:version/version.dart';
import '../migration.dart';
class Migration_0_0_1 implements MigrationFunc {
@override
Version get version => Version(0, 0, 1);
@override
Future<void> execute(FirestoreApi firestoreApi, Console console) async {
// [[ YOUR_MIGRATION_CODE_HERE ]]
}
}
Above code is a basic structure of a migration file.
- You implements
MigrationFunc(inlib/migrations/migration.dart) - You override
versionandexecutefunction. -
executefunction takes two parameters,- firestoreApi
- from
firebaseapispackage - use directly with the help of official document, or use prebuilt utils which I’ll explain below
- from
- console
- from
dart_consolepackage - use when you write some text(success, error etc)
- from
- firestoreApi
In order to modify your Firestore scheme, I made util classes (CRUDDocument and CRUDField). I only created some methods that I need for now. You can freely add other method. Maybe if you think it has general use cases, then please make a PR, so that others may get your help.
Sample code I wrote is something like below.
...
final crudField = CRUDField(
firestoreApi: firestoreApi,
console: console,
projectId: projectId,
);
final crudDocument = CRUDDocument(
firestoreApi: firestoreApi,
console: console,
projectId: projectId,
);
final documents = await crudDocument.readDocuments('test');
for (var document in documents) {
await crudField.createField(
document, 'hello', Value(stringValue: 'world'));
}
console.writeLine('done!');
...
Explain :
- Read collection called
test - Create field called
hellowith string value “world” to each document undertestcollection. - Print “done!” in console
4. Run it
Now you can run it. But before, It’s better for you to read basic commands and options in this application.
firestore_migration
A Dart console application for firestore migration
[Commands]
migration : Start migration work
[Options]
--version, -v
: Specify migration file version to run. Format should be match to semver(x.y.z) Default is latest.
--service-account, -s
: Path of service account json file. (required option)
[Flags]
--help : Display helper text on terminal
Now, You can run it.
dart run ./bin/firestore_migration.dart migrate -s service_account.json -v 0.0.1
And if you got this, then the job you asked for is done! Check your Firestore in Firebase console to see if data changed correctly.
Start : Firestore Migration...
this is 0.0.1
done!
Finish : Firestore Migration!
Conclusion
Note that you need to test it on the test project is safe. And I strongly recommend you to backup first before make any changes with this application.
I hope that this repository can help with those who has this specific need.
Any new feature idea or PR is very welcomed.
Cheers!
Latest comments (0)