Making well-behaved programs in Go with cybozu-go/cmd

We at kintone.com are using Go for variety of tasks.

In early days when we started using Go, there were only few in-house rules to create commands. As a result, we faced several difficulties:

• Some servers did not record access logs.
• Some programs did not record command execution logs.
• No common spec to handle SIGINT or SIGTERM.
• Some programs did not reopen log files, making log rotation difficult.
• Inconsistent log formats.

How to hack Go's http.Server correctly

One of my recent work was a design and implementation of a framework for Go programs. It is open-sourced at GitHub https://github.com/cybozu-go/cmd .

During the work, I have tackled how to enhance Go's http.Server by adding access logging and ability to stop gracefully. This article summarizes my findings and tricks used in github.com/cybozu-go/cmd package.

Introducing go-apt-cacher and go-apt-mirror

go-apt-cacher is a caching reverse-proxy designed specially for Debian/Ubuntu repositories. As it is written in Go, go-apt-cacher tolerates thousands of concurrent client connections and is very fast.

go-apt-mirror is a mirroring tool for Debian/Ubuntu repositories similar to apt-mirror. The biggest advantage of using go-apt-mirror is that it will never create incomplete/inconsistent mirrors.

This article describes our backgrounds and motivation as well as the design of these tools. They are available at https://github.com/cybozu-go/aptutil.

Transparent SOCKS proxy in Go to replace NAT

I am working behind a cloud service kintone.com as an infrastructure engineer. My recent work was the replacement of NAT inside our data center with a transparent SOCKS proxy.

In this post, I will describe our motivation for the replacement and how we can do it using Go effectively.

TL; DR

• NAT has deficiencies in routing, connectivity, and access control.
• We made a transparent SOCKS proxy transocks to replace NAT.
• Using iptables is far better than LD_PRELOAD for transparent proxy.
• We made our own SOCKS5 server usocksd too.

Split a string in C++11

The example below works just like PHP's explode.

std::vector<std::string> tokenize(const std::string& s, char c) {
    auto end = s.cend();
    auto start = end;

    std::vector<std::string> v;
    for( auto it = s.cbegin(); it != end; ++it ) {
        if( *it != c ) {
            if( start == end )
                start = it;
            continue;
        }
        if( start != end ) {
            v.emplace_back(start, it);
            start = end;
        }
    }
    if( start != end )
        v.emplace_back(start, end);
    return v;
}

Usage:

auto v = tokenize(" abc  def ", ' ');

v == std::vector<std::string>{"abc", "def"}; // true

Simple and efficient.