A place to share my gained knowledge and experience

"Life is all about sharing. If we are good at something, pass it on." - Mary Berry

Do not use jq when working with large number

2021-09-07

We use etcd to store application configuration. On production, config is loaded from json file by using Python.

I’m wondering if we can use jq to do that. So, I tried something like this:

host=${ETCD_HOST:-127.0.0.1}
port=${ETCD_PORT:-2379}
for key in $(jq -r 'keys[]' "$1"); do
  value=$(jq -r ".$key" -c "$1")
  ETCDCTL_API=3 etcdctl --endpoints="$host:$port" put "$key" "$value"
done

Config is loaded into etcd but some values are not the same as in the JSON file. What is going on?

Automate office tasks with Python

2021-06-10

Hàng tháng bạn nhận được một file excel về danh sách D. Từ danh sách này bạn cần filter cột C theo tiêu chí T1 để lấy ra danh sách D1. Có được D1 rồi bạn Save As thành t1.xlsx. Làm xong T1, bạn tiếp tục với T2 -> t2.xlsx … Làm tiếp cho đến Tn -> tn.xlsx. Có được các danh sách t1.xlsx -> tn.xlsx này rồi, giờ bạn cần gửi mail:

t1.xlsx: gửi cho P1, P2, P3

How to add full text search to my static website?

2020-12-30

After building the website, I want to add search function. There are some ways to do it:

ElasticSearch: setup and maintenance cost
MongoDB: have to insert posts into database
Google Custom Search Engine: ads?

So, is there any search engine written in Go?

Bleve
riot

Bleve has more star and be maintained more often than riot, I would like to give it a try first.

Backend

Looking at the documentation, there are three steps to add search to your website:

How to perform integration testing in Go?

2020-09-29

Integration testing can be triggered by using Drone downstream plugin:

steps:
- name: trigger
  image: plugins/downstream:linux-amd64
  settings:
    params:
    - COMMIT_BRANCH=${DRONE_COMMIT_BRANCH}
    repositories:
    - repo/integration-test@${DRONE_COMMIT_BRANCH}
    server: https://drone.example.com
    token:
      from_secret: drone_token

It can be separated with unit tests by using build tags:

// +build integration

package webserver_test

Then we can write code to perform integration test as usual.

Save draft mail in Zimbra web client using ChromeDP

2020-07-03

As an engineer, I want to automate everything as much as possible. This CLI tool is created to save a draft mail in Zimbra web client.

Read config file:

func initConfig() {
	if cfgFile != "" {
		// Use config file from the flag.
		viper.SetConfigFile(cfgFile)
	} else {
		// Find home directory.
		home, err := homedir.Dir()
		if err != nil {
			fmt.Println(err)
			os.Exit(1)
		}

		// Search config in home directory with name ".zwc" (without extension).
		viper.AddConfigPath(home)
		viper.SetConfigName(".zwc")
	}

	viper.AutomaticEnv() // read in environment variables that match

	// If a config file is found, read it in.
	if err := viper.ReadInConfig(); err != nil {
		log.Fatal(err)
	}

	fmt.Println("Using config file:", viper.ConfigFileUsed())
}

Google Calendar CLI

2020-04-13

Our company allows us to work from home some days a week. To do that, we have to create an event in Google Calendar.

I created this tool to run it from CLI.

First, take a look at this quickstart.

Create initical code by running:

$ cobra init

$ tree -L 2
.
├── LICENSE
├── cmd
│   └── root.go
├── main.go

Create event command:

$ cobra add event
$ cobra add insert -p 'eventCmd'

$ tree -L 2
.
├── LICENSE
├── cmd
│   ├── event.go
│   ├── event_insert.go
│   └── root.go
├── main.go

go/packages.Load: no packages found

2019-10-22

I have a mono-repo, structure like this:

repo
├── service1
├── service2
├── go.mod

I often open the root folder from the command line by using code .. After that I open a file in service1, and cannot go to definition. Here’s the logs from gopls:

[Info  - 5:49:28 AM] 2019/10/22 05:49:28 20.675656ms for GOROOT=/usr/local/Cellar/go/1.13/libexec GOPATH=/Users/quanta/go GO111MODULE=auto PWD=/Users/quanta/go/src/github.com/owner/repo go "list" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "--" "/Users/quanta/go/src/github.com/owner/repo/a/b/c", stderr: <<go: directory a/b/c is outside main module
>>

[Error - 5:49:32 AM] Request textDocument/definition failed.
  Message: go/packages.Load: no packages found for /Users/quanta/go/src/github.com/owner/repo/a/b/c/file.go
  Code: 0

database/sql: never ignore errors

2019-10-16

I’m reading Building RESTful Web Services with Go. And in chapter 4, there is an example to play around with SQLite:

db, err := sql.Open("sqlite3", "./books.db")
if err != nil {
	log.Fatal(err)
}

statement, err := db.Prepare("CREATE TABLE IF NOT EXISTS books (id INTERGER PRIMARY KEY, isbn INTEGER, author VARCHAR(64), name VARCHAR(64) NULL)")
if err != nil {
	log.Fatal(err)
} else {
	log.Println("Created table books successfully")
}
statement.Exec()

statement, err = db.Prepare("INSERT INTO books (name, author, isbn) VALUES (?, ?, ?)")
if err != nil {
	log.Fatal(err)
}
statement.Exec("Life is a joke", "The Javna brothers", 123456789)
log.Println("Inserted first book into db")
rows, err := db.Query("SELECT id, name, author FROM books")
var tempBook Book
for rows.Next() {
	rows.Scan(&tempBook.id, &tempBook.name, &tempBook.author)
	log.Printf("ID: %d, Book: %s, Author: %s\n", tempBook.id, tempBook.name, tempBook.author)
}

How do I build this blog?

2019-10-11

Raspberry Pi 4

Recently, I decided to find a new job as a Golang developer. So, I updated my resume, sent to my friends to ask for review. Then I submitted it enclosed herewith a cover letter to recruiters. Some didn’t reply, and the other replied with a message like this “You are so good, but I’m so sorry…”.

What is the reason?

As you can see in my resume, I started my career as a .NET developer, then my passionate on Linux and open source lead me to a different direction: system administrator. I dedicated myself to this role for a significant period before transitioning back to work as a Golang developer 2 years ago.

Auto reload your Go webserver with Gulp

2019-09-20

When you developp a webserver with Go, you must compile each time you do an update in your code. Well.. this is redundant. With Gulp you can automatize this task… Indeed, when a go file is modified, a task compile the application in the “bin” folder (“gopath/bin”) then another launch the executable (the webserver).

https://medium.com/@etiennerouzeaud/autoreload-your-go-webserver-with-gulp-ee5e231d133d

const gulp     = require('gulp'),
      util     = require('gulp-util'),
      notifier = require('node-notifier'),
      child    = require('child_process'),
      os       = require('os'),
      path     = require('path');

var server = 'null'

function build() {
    var build = child.spawn('go', ['install']);

    build.stdout.on('data', (data) => {
        console.log(`stdout: ${data}`);
    });

    build.stderr.on('data', (data) => {
        console.error(`stderr: ${data}`);
    });

    return build;
}

function spawn(done) {
    if (server && server != 'null') {
        server.kill();
    }

    var path_folder = process.cwd().split(path.sep)
    var length = path_folder.length
    var app = path_folder[length - parseInt(1)];

    if (os.platform() == 'win32') {
        server = child.spawn(app + '.exe')
    } else {
        server = child.spawn(app)
    }

    server.stdout.on('data', (data) => {
        console.log(`stdout: ${data}`);
    });

    server.stderr.on('data', (data) => {
        console.log(`stderr: ${data}`);
    });

    done();
}

const serve = gulp.series(build, spawn)
function watch(done) {
    gulp.watch(['*.go', '**/*.go'], serve);
    done();
}

exports.serve = serve
exports.watch = watch
exports.default = gulp.parallel(serve, watch)

First
«
1
2
3
»
Last