It is basically default property of a framework.
The actions you perform on daily bases symfony create a cache for that so that it will fetch data from store cache data and logs and other things is part of this.

Yes you can delete cache folder as well it and it will again create for you.