Client Side manipulation of ObjectPicker

I’d like to remove options from an ObjectPicker without having to reload it. I tried removing the options from the select and then trigger a “chosen:updated” event, but that does not seem to have any effect.
Can someone guide me on how to update the ObjectPicker without an AjaxRequest?

Thank you!