Tensorflow: Difference Between Functioning Of __call__() And Call() When Designing Custom Layers?

I've designed a neural network with a custom layer in TensorFlow. Firstly, I used __call__() to define the forward pass and then I called the model.summary(). This gave me followin

Solution 1:

The ValueError you've faced is reasonable. This is asked and get answered already over here And also the difference between call and __call__ is also answered already over here;(as you didn't give any code to reproduce so I'm referring to this link). However, FYI, I've written an article about model subclassing and custom training from scratch, I hope the modeling approach part may give you some insight (the article link).

Update

Too many questions in the comment section, so I like to add my response here. Basically, the __call__ method is implemented in the Layer class. And this layer class is inherited by the Model class. Let's say

import inspect
from tensorflow.keras.models import Model

# inspect.getmro(CLASS) returns a tuple of class CLASS's base classes, # including CLASS, in Method Resolution Order (MRO).
inspect.getmro(Model)

# out
(tensorflow.python.keras.engine.training.Model,
tensorflow.python.keras.engine.base_layer.Layer,...)

Now, in __call__ method in the Layer class has assigned self.call method to call_fn. Which is further used here. Such as:

def__call__(self, input):
    call_fn = self.call
    outputs = call_fn(input, ...)
    return outputs

So when we call our Model class instance, it's inherited __call__ method (from Layer class) will be executed automatically, which calls our own call method.

Model Summary

The modeling with Subclassing API is not as same as Sequential or Functional API. It is not graph networks and therefore only a node containing the model name would be plotted with these models. We can't really assume anything about the structure of a subclassed Model. That's why we can't get the output shape. However, there's a workaround to get the facilities of model shape summary and plot it either conveniently. Check here.