I’m a self taught developer just in my teens about to start my first major project for myself. I’m looking for architecture advice as I want to do this the right way.

I’m worried I’m overcomplicating this by building it myself when something like gRPC can maybe solve this?

I’m looking for a way to have many applications all scattered across individual servers, and check the status of the process as well as communicate with it (start, stop, restart, instruct). It must have REST / HTTP access from my golang API.

Proposed (existing way): A master TCP socket server which each process connects to as a client. Services then communicate directly with master only, which forwards it to the corresponding app.

Issues: Would need to write an additional communication layor for HTTP / REST access from golang API. Process could be accidentally ran without a valid connection. Lots of low level network instruct / complexity involved.

Performance: It needs to be performant (can communicate inexpensively and quickly) as it will have a large load of messages as I am also planning on sending log messages to the master.

How can I improve upon this?

Does something exist already?

Could gRPC be better than a full blown socket server?

SSH Proposal: Some people have suggested using SSH. While this seems like a good idea, from a security standpoint it could introduce a lot of complexity on the configuration side and require constant maintenance to ensure each service communicating with the individual server has the necessary authorized keys etc. I’m interested to here advice on this one.