The rivet looking pin is called a "drive screw" and it has spiral flutes on the sides so when it is pressed in, the flutes dig into the material. IF you could simply pull it out, you would use a rotation motion but not complete rotations like a screw. The only way to remove it is to carefully grind off the head to remove the bracket. To protect your surface, you can use some thick tape such as a couple of layers of duct tape. Use a small burr or wheel to slowly remove the head. After you have the head removed, the bracket should pry off. Now, you need to cut the screw flush with the surface OR perhaps you can grab the screw body and twist it out. Usually, they have right hand flutes so CCW rotation gets them out. Look up Drive Screws on the McMaster web page for more info.