Had a discussion today in how to implement services that work with messages coming in from event queues. We call these services processors. One of us argues for using several functions, while the other argues for fewer functions that do more things, according to idiomatic go. To make things more difficult, there’s a healthy amount of async stuff happening and I’m new to async.

In essence, what the code must do is:

• set up the project: config, logging and start the service up.
• listening to message queues, potentially respond. Every time a message pops in, we may start to spin up a new goroutine.
• process some of the data inside the message, potentially return it to the user.

I cannot just copy the code for legal reasons but will make pseudo code.
One approach consists of making two functions, main() and do().

main sets up config, logger, and service.
do listens to message queue. Because of NATS library, the code that handles the message is a callback function. It looks something like this:

sub, err := nats.Subscribe(topic, cbFunction)

This approach insists in making an anonymous function that peels the message for the field that we want to process, processes it if it’s synchronous, and responds to the queue with a new message with a field containing this processed data.

# inside do()

sub, err := nc.Subscribe(topic, func(msg *nats.Msg) {
  // logs
  // checks data consistency
  // processes data
  // creates response with processed data inside
  // responds to queue with processed data
})

If it’s async, then uses a channel to send data, and the part that processes it stays in it’s own goroutine, listening to the channel.

# inside do()

go func() {
  for {
    select {
      // listens to channel and processes sent in data
      // handles errors if any
      // responds to queue
    }
  }
}()

sub, err := nc.Subscribe(topic, func(msg *nats.Msg) {
  // logs
  // checks data consistency
  // sends data to channel
})

There is another completely different approach, which is using newly created functions handleMessage(msg *nats.Msg) and process(data).

# outside do()

func handleMessage(msg *nats.Msg) {
  // logs
  // checks data consistency
  processedData, err := process(data)
  // creates response with processed data inside
  // responds to queue with processed data
}

func process(data) (processedData, error) {
  // processes data
  // returns processedData
}


# inside do()
sub, err := nc.Subscribe(topic, func(msg *nats.Msg) {
  handleMessage(msg)
})

if needs to be async, then process is invoked in do() and handleMessage doesn’t call process directly, but rather uses channel. In turn, we have a second channel for returning data and potentially a third one for error. So:

# outside do()

func handleMessage(ctx, chIn, chOut, chErr, msg) {
  // logs
  // checks data consistency
  processedData, err := process(data)
  // handles errors or creates response with processed data inside
  // responds to queue with processed data
}

func process(ctx, chIn, chOut, chErr) (processedData, error) {
  // processes data
  // returns processedData
}


# inside do()
chIn := make(chan []byte)
chOut := make(chan []byte)
chErr := make(chan error)

go process(ctx, chIn, chOut, chErr)

sub, err := nc.Subscribe(topic, func(msg *nats.Msg) {
  go handleMessage(ctx, chIn, chOut, chErr, msg)
})

Right off the bat, which one is the best approach? Are there better ways of doing this?

Another big consideration: some of these processors are going to be an example for customers, who are also programmers. We’re going to have examples in different programming languages. One side argue for all examples having the same structure. The other one for being idiomatic to each language. As a go dev, which example would you prefer and is clearer to grasp?

Other considerations that both side argue in their favour:

• named functions: more simple, more testable, more reusable – processors would mostly differ in how they process the data.
• anon functions: more simple, named function has too many channels, idiomatic go – fine for something like Python, but not go.

I expected to put up a case but he put a different one. Both have their merits and weaknesses, I guess. Which one would you prefer as go devs? Are we missing something? Should we be idiomatic or agnostic? Are three channels that much of a problem?