Is your oven placed outside? Is it possible of getting wet? If yes, it may be a problem!
Even if your oven is made out of cob or brick it will cause a problem. Cob ovens are made out of clay so if it gets wet on a long period of time it may begin to break. We know that to form an image or thing using clay, we have to mold them. As your cob oven gets wet it may deform because the clay may soften and loose its structural strength. Some of its part may also fall away, and it is possible also that the entire cob oven will collapse.
The wall of cob and brick ovens seep the water overtime. The problem with the brick oven is that if you didn't dry the whole oven totally, in putting fire on it again, heat will turn into steam and it will attempt to escape. The pressure from the steam may cause the walls of the oven to crack. How will it happen? The walls of the brick oven is moist when it is not totally dry that's why.
The best solution for this problem is using a rain cover. It should be made from waterproof materials so the water will not touch the oven.