A simple approach is to have your manager track your objects with weak references. Weak references allow you to keep a reference to an object but the weak reference won’t prevent it from being garbage collected.

var reference = new WeakReference<SomeType>(myObjectToManage);

// later
SomeType value;
if (reference.TryGetTarget(out value)) {
    // the object referenced is still alive.
}
else {
    // the object referenced has been garbage collected.
}

The only downside of this approach is the objects may live for a while after there are no references to them, until a garbage collection occurs.

Another option would be to implement your own reference counting scheme. This requires some discipline in the client code that is using these objects to make sure references are tracked correctly.

This is a pretty broad question, further complicated by your not terribly well-defined performance sensitivities. However, fundamentally, you have a few choices:

Have the manager act as factory to hand out wrappers. Wrappers are then collected like regular objects (via GC). Wrappers implement a finalizer that informs the manager that the underlying wrapped resource is being released. Thus the manager can count up (on handing out wrappers) and down (on wrapper finalizer run) to release the underlying resources.

This kind of simple counting assumes that the resources are external, or at least acyclic.

If cycles among resources are possible, then additional handling must be employed, which might be periodic scanning of the references (e.g. your own mini-GC).

Use the Resource Acquisition Is Initialization, pattern, as adapted to C# pattern the using statement and IDisposable. This also often results in wrappers. See Link

Note that in C# you can use structs for lightweight wrappers; however, not if you rely on finalizers.

And as the other @Erik notes, we can use the C# built-in weak reference classes. From MSDN:

A weak reference allows the garbage collector to collect an object while still allowing an application to access the object. If you need the object, you can still obtain a strong reference to it and prevent it from being collected. For more information about how to use short and long weak references, see Weak References.

There’s no straightforward way for garbage-collected languages like Java or C#. By getting rid of deterministic memory management/RAII, they ironically made resource management more complicated if your resource isn’t memory and if you want to encapsulate resource management as far as possible.

To reduce the need for manual resource management, C# has using () blocks, Java has try-with-resource. You can perhaps introduce an object representing a handle to the actual object. When the handle is closed, ownership of the object returns to the pool. By creating the handle in a using-block, you don’t have to release ownership manually as the language takes care of that. For complex shared ownership, this is not sufficient and you’ll have to resort to entirely manual resource management.

Trying to hack around the GC system with techniques like the flyweight pattern is often not worth the effort. The additional indirection, bookkeeping, and possible garbage from temporary objects associated with such a solution may outweigh the benefits of an object pool. Be sure to measure which is better, instead of settling on one solution prematurely.

Having a look at this and this it seems that you need to implement IDisposable in your classes to notify the manager object everytime one of the instances is destroyed, you should also implement your own reference counting system in the manager, a simple factory should do.

So your managed classes will be something like this:

   public class SharedResourceWrapper: IDisposable {

    private Manager manager; // a handle to the manager

    public SharedResourceWrapper(Manager manager){
        this.manager = ... // allocates the resource
    }

    public void Dispose(){
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing){
        if (disposing) {
            manager.RemoveReference();
        }
    }
}

And your Manager class should have this two methods

public SharedResourceWrapper Create(){
  var newInstance = new SharedResourceWrapper(this)
   references++;
}

 public void RemoveReference() {
    ...
    reference--;
    if( reference == 0){
     ....
    }
   }

Dispose Pattern

When a class controls the lifetime of unmanaged resources you should at a minimum implement IDisposable. This allows callers to deterministically release the unmanaged resource by either calling Dispose or with a using statement:

using (var unmanaged = GetUnmanagedThing())
{
    // Doing stuff with unmanaged
}

Implementing IDisposable is relatively straightforward. In your case each instance should be constructed with a reference to the manager and should call the manager when they are disposed. Make sure to follow these guidelines:

• Dispose can be called multiple times for a single object, make sure your Dispose method handles this.
• Don’t throw exceptions from Dispose unless the entire process is on fire.
• Make sure to stop methods on the object from being used when it has been disposed, normally by throwing ObjectDisposedException.

The MSDN Dispose Pattern Article goes into a lot more detail. That page advises you to implement dispose logic in a virtual Dispose(bool disposing) method as this allows finalizers and derived classes to correctly handle object disposal. If your object does not have any inheritors and does not use finalization then this pattern is unnecessary and you can just implement the Dispose method directly.

Finalization

Ensuring that unmanaged resources are destroyed when an object is garbage collected (without having been disposed first) involves finalizers, which are notoriously difficult to implement correctly because everything you know is wrong. In particular, objects can become available for collection sooner than you think, for example an object can become eligible for collection during execution of a method on that very object. Care must therefore be taken to ensure managed resources are not disposed while they are still in use.

Finalizers only run after an object is garbage collected, which is non-deterministic. There may be a lengthy delay between when an object is no longer referenced and when it is finalized which means a lengthy delay before your unmanaged resources are destroyed. In addition finalizers result in a performance penalties as the finalizer must be scheduled to run and the memory for the object cannot be reclaimed until the finalizer has finished running.

If your application is performance sensitive then you should aim to always dispose unmanaged resources using the Dispose pattern, which makes finalization unnecessary. Therefore my advice would be to avoid relying on finalization and instead ensure all parts of your application correctly use the dispose pattern. You can add logging to your finalizer to ensure warn when the dispose pattern is not correctly followed:

public void Dispose() {
    // Implement dispose here....
    GC.SuppressFinalize(this);
}

~UnmanagedThing() {
    Log.Warning("Unmanaged thing not properly disposed");
}

If this solution isn’t suitable then you may be able to use a SafeHandle to dispose of unmanaged resources.

No really, I want to write a finalizer

The general idea is to ensure that your object is unreachable, e.g. by having your manager hold only a weak reference to your object, or by having your manager hold a reference to a nested object. If your object becomes eligible for destruction then the finalizer will (hopefully) be called and can be used to release your unmanaged resources.