I don't do much with IMAP so am not sure I can help but it sounds like it might be helpful for you to be able to determine whether a folder is local. To do this, right click on the folder name then select "Folder information".
If you find a local folder you will see a filename in the form of FOLXXXX.PMM. If it is a folder that you are trying to remove you can remove the associated files files from the home mailbox directory. They will be the referenced .PMM file and an associated PMI file. With Pegasus Mail closed, use a file explorer to move them out of the mailbox directory.
Another thought is that the folder won't delete because it's not empty. I don't know if this thought has any merit in the IMAP world; I just throwing it out there.
One other thing that might be worth a try is to rebuild the HIERARCH.PM file which is where the folder list is stored. This will clear the folder list of any invalid entries but won't help if the folder physically exists. WARNING: You will lose all folder structure if you do this but can undo the process if you have a copy to revert back to. If you choose to pursue this, with Pegasus Mail closed, change the name of the HIERARCH.PM file to something like HIERARCH_PM.SAVED, start Pegasus Mail. A new HIERARCH.PM file will be created automatically.
To undo, replace the new HIERARCH.PM with the original renamed one (Pegasus Mail must be closed).