Spaces:

AccelerationConsortium

/

cobot280pi-gradio

like 0

Sleeping

is the gradio api intended to be used? in which cases would this be used instead of the regular python api?

The main idea here is to use the gradio API via a Colab notebook. This primarily relates back to https://github.com/AccelerationConsortium/ac-training-lab/issues/77. This way, we allow people to access the Cobot programmatically without exposing the HiveMQ credentials. I.e., create a setting "as if they had the Cobot in front of them" but with the right guardrails in place.

I also just realized this might conflict with https://huggingface.co/spaces/AccelerationConsortium/cobot280pi-gradio/discussions/2#672e963504a0498c6aea5226. Not sure if it's feasible to keep the original API endpoints mostly intact and have the "call query camera anytime the robot moves" be specifically linked to button clicking rather than the under the hood API. If it's too much of a pain to implement, I'd lean towards reverting the "call query camera anytime the robot moves" so we can keep the more authentic programmatic usage experience.

Open to thoughts.

Maybe programmatically triggering button clicks since usage like https://huggingface.co/spaces/AccelerationConsortium/cobot280pi-gradio/blob/1de63fb1edb9171efd3e7c0f9a1922e7126e0a48/app.py#L274-L279 is what directly impacts the API.

See also https://github.com/gradio-app/gradio/issues/7348#issuecomment-1946320538

Again, if it's a pain we can revert (sorry!) and put it on a backlog.

Sure, ive never used the Gradio API before, so ill play around with it for a bit. If theres a way to distinguish API access vs accessing the website, it may be possible to update camera view only in the website version. Would that be a good solution?

Yes, that would be great if feasible - if too much of a pain, no worries.

so after looking at it for a while it seems that it is possible to differentiate programmatic access from the website version but it would require adding an additional parameter to each api endpoint function. This seems a bit inelegant so I just decided to remove the auto image refresh feature.

I have refactored the code slightly to make the API access a bit cleaner. I've also created a demo notebook for using the cobot through the API access here https://github.com/AccelerationConsortium/ac-training-lab/blob/cobot-usage-docs/src/ac_training_lab/cobot280pi/gradio-client-demo.ipynb

There doesn't seem to be a way to change the function name from predict to anything else, but the refactors made it so that each endpoint doesn't have to be given all 18 parameters, which does simplify the API access significantly.

I'm ok with the removal of the refresh feature. Thanks for digging.

I think I came across a kwarg somewhere that let you change the name from the default of predict, however it seemed like that applied across all endpoints. Assuming that's true, thoughts on an alternative to "predict"? Maybe just "cobot"?

Awesome to see the notebook! Nice job. For the notebook, maybe add a cell after sending the position and a cell after setting the gripper state showing the queried values? (I.e., verify that it went relatively according to plan). Likewise, a queried image at the very end to point out visually that the movement occurred.

Minor point "the cell below" instead of "this cell"

Minor "The cell prints ..." can probably be removed except for the first instance. Maybe say "The cells below print ..."

Maybe add a picture of the cobot at the beginning? When the livestream is robust, maybe we could include a link or embed in the notebook.

Consider adding a diagram of the angles before or after the first query if one is easily available.

Aside: is there a way to query joint angles in the HF API you implemented?

I found the spot I was thinking of. It's a kwarg called api_name within Interface; however, I see that this one uses Blocks, which doesn't have the same kwarg.

EDIT: https://www.google.com/search?q=gradio+blocks+api_name → https://www.gradio.app/guides/using-blocks-like-functions → (I noticed) translate_btn.click(translate, inputs=english, outputs=german, api_name="translate-to-german") and realized it may be on a per-button basis. It's in the click method, which shows api_name as a kwarg. However, it looks like maybe it's still called via predict.. Maybe not feasible like you mentioned.

Yep, the first query cell queries the joint angles.

api_name is a kwarg in the button.click function but also in client.predict, so those two map to each other. As far as i could tell there is no real way to change the predict function to anything else. We could have a small wrapper around client in the notebook, but im not sure i see the benefit.

Ok, cool. I think I finally caught up 😅

Agreed. Thanks!