Unshare a Library from User

I’m trying to use the Web API to unshare a library, I have a working token and the correct Repo-Id however upon making the request I simply get “permission denied” as a response, I’ve tried this from both a normal and an administrator account. What am I doing wrong?