I use golang for processing large JSON files(currently about 30GB), golang is useful for development, but it seems to be much slower and use much more memory than I expected.
So I tested performance of interface {} with about 200MB text file. It contains 10M entries having one key value pair separated with "\t".
So I simply try to use bytes.Split and strings.Split, but my framework processing arbitrary object requires data whose type is encoding/json's one. So every time I should copy []string to []interface {} as noted in many golang-related documents as the following:

// Please forgive me dirty error processing...
b, err := ioutil.ReadFile("../10M.txt")
if err != nil {
    panic(err)
}
lines := bytes.Split(b, []byte("\n"))
objs := make([]interface {}, len(lines))
for i, line := range lines {
    objs[i] = string(line)
}
lines = nil
for i, obj := range objs {
    parts := strings.Split(obj.(string), "")
    iparts := make([]interface {}, len(parts))
    for i, part := range parts {
        iparts[i] = part
    }
    objs[i] = iparts
}

On my Windows 8.1 (64bit) laptop, this takes >=2GB memory and >=1min (In the same environment, I can count value with easy Unix commands in 22s.)

At first, I can't believe this result, because even if I should copy array to convert types (I can agree with this process to process arbitrary type, because it happens only at first time loading and parsing), this program takes about 2*(64/8)*10MB for []interface {}.

I read some code about runtime in src/runtime/iface.go and src/runtime/runtime2.go and some disassembled code with go tool objdump, I observed mysterious memory copy at assigning part to iparts and returning this iparts.

What I expected are,

• interface {} is like C struct having pointers of type and data. So when I allocate []interface {} with appropriate size, we can set interface entry without calling runtime functions.
• returning interface {} can be done with using 2 registers or stack, so we can avoid memory allocation for it.
• string are immutable and []byte are mutable, so converting between them requires memory copy but I want to avoid this as shown in this example http://qiita.com/mattn/items/176459728ff4f854b165 (Sorry, this is Japanese document). And I use this technique in unmarshaling JSON []byte from file.

Of course, this is based on my poor golang experience, so if someone knows about this behavior, please tell me about this! I think golang is best for developing rapidly safe and fast programs as used in some CoreOS-related projects. So if someone knows how to process arbitrary object without using interface {}, please tell me about this too!