Unlike many other commodity futures contracts, oil is "physically delivered". Anyone holding an open position as of the "first notice date" for each settled contract month is at risk of getting a "delivery notice". A delivery notice, either issued by a long contract holder, or received by a short contract holder is on the hook to either deliver or receive 1000 bbls of oil.

 

Most brokers won't let you hold an open oil contract past the first notice date. They don't want to be involved in actual oil delivery.

 

So, to answer your question, shorts have to cover each and every month unless they are a legitimate oil user (refinery, tank farm, etc.). 

 

Unless demand drops just as fast (or faster) than supply, the price is going to rise. Opec is unofficially using price as a measure of their success in balancing S/D.