bioclip-amazing-demo - REX & HowTo make some API call from NodeJS application

#7
by 3oly - opened

hi here πŸ‘‹

FIRST I would like to tell you that this demo is amazing ! It's great work.πŸ‘πŸ‘

In my current context (a small study in fact :p), this demo has currently the very-first position. ^^

This said, I have some feedbacks πŸ‘€, improvement ideas πŸ’‘, and questions ❓

  • A) First I'd reported demo broken here πŸ‘€ and the issue was fix mega-quickly πŸ‘πŸ‘,
    I think as improvement, it could be great to have some kind of link/button with "report malfunction" label πŸ’‘ close to the demo to know where to report issue/improvement idea. that's so minor.

  • B) I jump quickly here in hugging face space of the demo (I'm novice with ML). And trying to see if it will be possible to call this demo via API, I saw HuggingFace API feature (cf. footer link "Use via API").

But I encoutner some confusion with πŸ‘€Node.js provided samples : out-of-erro they are simply NOT working at all ❌.
So before leaving the idea, I had to switch on πŸ‘€ Bash very-simple examples and Bash was a successβœ”οΈ.

Finally with some try/error/repeat cycles, I finally find the good way to call this API via NodeJS client.
I provide a public repository πŸ‘€ to show you my final tests : #1 of bioclipBootstrap

  • C) now I'm able to query this demo API, I would like to know the term of use, and I'm asking myself if I could freely integrate them in a thir party (Bluesky bot) application for example (query rate is really low: ex. ~5 queries per day) with related paper reference in my project page ❓

  • D) I dont know if some improvements are planed but here is some REX/raw ideas I'had while testing this API

in the API response :

  • πŸ’‘ add a separate field for Species/Genus/Family.. (instead of an huge string with all of them),
  • πŸ’‘ add a separate field for "Common name" (ie. current label in parenthesis when present)
    I think that's not quit easy in ML context where it seems that "all is label" but if there is a logic (ie. space, upercase, parenthesis) to apply to separate fields, this must be done server side.
  • πŸ’‘ add a sample related image url for each prediction dataset : this would be great to think of prediction result and live compare current image used with predicted image sample (in addition to the confident ratio).
  • πŸ’‘add a way to get more context about a given species result, for example a link to avibase that give you a path to extends your knowledge on a given entry. (not that easy I think)

Best regards
Brice

HDR Imageomics Institute org

Thanks for the feedback! Some answers to your questions/thoughts:

  • Generally, we'd expect errors in the demo to be reported here (in the discussions tab), but the GitHub was also a good choice (glad you found it!).
  • We do have a package for python and CLI interaction with BioCLIP: pybioclip.
  • We are indeed working on some more upgrades, for instance a sample image + more info link (to the EOL taxa page) update is coming very soon for open-ended classification.

For the separate field options, do you mean only return that particular level instead of the full string? For instance, with "common name" as a separate field, do you mean the option to return only the common name without the taxonomic information? Both would be doable with the returned string. Or are you asking about a dictionary-style return (this can be achieved with pybioclip)?

HDR Imageomics Institute org

Just some additional things to consider @3oly :

  • We are not currently planning to upgrade the HF space to premium (paid). Therefore, the resources (and access limits) available for the demo are determined by HF's free tier.
  • We're committed to making the trained model more easily usable within the Python ecosystem (see pybioclip as per above) but at least for now not under the NodeJS ecosystem or as a standalone API server. (This could change in the future.) However, to set up an API server yourself under Python the pybioclip package should provide everything you need (and Python has a great web stack to build with).
  • Pybioclip is under active development and feature requests are most welcome.

Thanks for all this detailed answer I'll then dig pythonclip ^^

you asking about a dictionary-style return (this can be achieved with pybioclip)

yes it seems that the client is already doing that.

[{
'kingdom': 'Animalia',
'phylum': 'Chordata',
'class': 'Mammalia',
'order': 'Carnivora',
'family': 'Ursidae',
'genus': 'Ursus',
'species_epithet': 'arctos',
'species': 'Ursus arctos',
'common_name': 'Kodiak bear'
'score': 0.9356034994125366
}]

regards

3oly changed discussion title from bioclip-amazin-demo - REX & HowTo make some API call from NodeJS application to bioclip-amazing-demo - REX & HowTo make some API call from NodeJS application

Hi again, following some tests here is my new feedback

the requirements to play with the client are very huge in term of disk space and that prevent me to maintain a dedicated endpoint simply (out of pay for additional resource).
but the client is well designed and easy to use :+1:
and the client fit my need with expected result representation

what I discover is that HF is automatically mapping a API endpoint when gr UI component is relying on a function !

For ex. on my fork, adding a api_classification method was not sufficent, but if I map it to UI component, then the new endpoints appears to the HF endpoints and there is a way to trigger it from external client (bash/nodejs...) without extra effort

image.png

(this explain why some Enpoints automatically listed by HF "API endpoint" footer "feature" are not working properly. On another way I were unable to find HF dedicated documentation)

I'm too bad at python and ML to create on my own HF fork a dedicated endpoint that produce the same result as my "try" function

image.png

MVP need:

  • input : image url + rank
  • output : list of predictions where a prediction is a score + previous dictionnary

I would like (if possible) to suggest an improvement here on the "imageomics/bioclip-demo" that add to the UI (and -auto- under the hood to the API endpoints)
a new button+form+result that answer this MVP need.

is this dicussion sufficent to suggest this or should I create a ticket somewhere ?

(what I suspect is that the data model here on the demo maybe contains only label and not structured result ?)

thanks for all
regards

I finally found gradio endpoint API doc https://www.gradio.app/guides/sharing-your-app#api-page
I've made a poc of api with json as output
https://huggingface.co/spaces/3oly/gradioAPI

Hi, I'm so happy...following pyBird MVP + re-using (inline in HF) some code of your pybioclip python client, I finally succeed to provide the expected API format πŸ˜„.

https://huggingface.co/spaces/3oly/grBird

Any feedbacks are welcome.

It would be great for you to integrate in another tab this kind of interaction.
But this new fork/demo solves this discussion origin as I'm able to progress on my side πŸ‘πŸ‘
Thanks again for your previous inputs that help me so much.πŸ‘

here is the very-first BioClip classification on bluesky app ^^

image.png

Sign up or log in to comment