It's possible to increase the queue size but let's talk about something here.
Why is your queue filling up? Because your producers are adding work items to the queue faster than the consumers can take them off the queue.
What would happen if you increase the queue size? The producers will still be producing at the rate they are currently producing, and the consumers will still be consuming at the rate they are currently consuming. This means that the queue will still fill up. Except now, the queue is bigger which will put more memory pressure on the system. In fact, this means the consumers might slow down, which might exacerbate the production rate greater than consumption rate problem. From there, a vicious feedback loop will form.
Queues are helpful for handling variable load. They are not helpful for handing scenarios where the production rate exceeds the consumption rate. In this scenario, a queue filling up is a form of back pressure. It is telling you that you need to slow down the rate of production (or speed up the rate of consumption).
Only increase the queue size if you are in a situation where you have variable load that sometimes exceeds capacity. Do not increase the queue size if you are in a situation where production rate exceeds consumption rate.