sample_embedding_folder2/0780640.txt

Ticket Name: RTOS: Emulator and TDA2x has different result with TIDL(version 01.01.03).

Query Text:
Other Parts Discussed in Thread: TDA2 Tool/software: TI-RTOS Hello, I have test my own model using TIDL(version 01.01.03). I recently found that the PC emulator and TDA2x generate different results. I have checked each layer dump , discovering a convolution layer gives different output. Furthermore, I checked the layer result which gives different output and found another strange phenomenon. My convolution layer has 96 input channels and 192 output channels with group = 8 and kernel size = 3. The output feature maps is different at 0th, 24th, 48th, 72th, 96th, 120th,144th, 178th. It seems that the first generated feature map in each group has different value. After that I extract the particular convolution layer to do the test which input is previous layer output.(ex: trace_dump7.y ). The result from PC emulator and TDA2x is different. I generate the TDA2x eve test and PC-emulator ,using below instruction respectively. "gmake TARGET_BUILD=release CORE=eve all" "gmake TARGET_BUILD=release TARGET_PLATFORM=PC CORE=eve all" All detail is in the appendix .rar file , including Convolution layer information , input data , PC_emulator result and TDA2x result. Thank you. TIDL_issue.rar

Responses:
The PC emualtion is expected to give the same result as Target. Your build commands are right. BTW, Is the PC emualtion results matching with expected results (caffe)? Which is the first layer mismatching ?

I just check the proto file. This has only one layer. We will look in to this get back on this .

The rar does not contain, the caffe model file? Can you share the caffemodel also?

The Caffe model contains only the layer which begins to generate different output.Layer_model.rar

We have checked the output which given by PC emulator. Detection Boxes generated by PC emulator are similar to the result from Caffe.

Hi KaiNan Hsieh, We could not re-produce this issue in latest code base. Looks line one the recent fix had solved this isse as well. We will locate the right fix and share the patch soon.

Hi KaiNan Hsieh, We are not able to reproduce this issue in 01.01.03 release as well. Are you observing this issue when you the full network or with the attached single layer as well?

We found this issue when we do the detection with full network , and we dumped each layer to check which layer gives the different output. Then we test the attached single layer see if it gives the same issue which we found in full network. So this issue was found in both cases. Thank you

Hello , We have tried another case, discovering that it has different output with convolution layer on PC-simulator and TDA2x. In our test case we found the PC-simulator and TDA2x have same output when we didn't use dense convolution. Therefore , We regenerate the model by set some layer using dense convolution , and these makes the output different. Is any problem with dense convolution may cause this problem? Thank youconvolution-Issue.rar

Hi, Last thread I have mentioned that dense convolution gives different output in some case. I have built a caffe model with only 3 convolution layer and test different setting of dense convolution.(The first model import with no dense convolution and another with all dense convolution). The results are in the attached file ,including files below: import.txt , infer.txt , caffemodel , deploy.prototxrt, input data, output data from TDA2 and PC-simulator. I used the command "gmake TARGET_BUILD=release TARGET_PLATFORM=PC CORE=eve all" to generate eve_release_out.exe and generate the PC-simulation result. Use "eve_release.out" which is existed in the testcv/out folder to generate the TDA2 result. Thank you, ti_conv_issue.rar

Hi KaiNan Hsieh, Could you please provide the version of your TIDL? I can get incorrect result with your eve_test_dl_algo.out.exe, but get correct result with TIDL 1.1.3. Please provide your import tool binary to make sure I can reproduce it. So you mean that PC-simulation is not align with the inference on TDA2 target? Your prototxt has 3 conv layers, only the third layer can be configured as dense conv. All 3 layers are not aligned or only the third layer is incorrect? Thanks & Best Regards! ZM

Hi, I can reproduce it now(first channel of each group of the third layer is not aligned with PC tool). and get back to you later. Thanks & Best Regards! ZM

Thanks for your reply. The output of these 3 layers have the same result with TDA2 and PC-simulation at first and second layers.But third layer is not align with TDA2. Only the third layer is not aligned. Thank you

Thanks, Problem is solved.