segformer-b3

This model is a fine-tuned version of nvidia/mit-b3 on the segments/sidewalk-semantic dataset. It achieves the following results on the evaluation set:

• Loss: 0.7826
• Mean Iou: 0.3995
• Mean Accuracy: 0.4977
• Overall Accuracy: 0.8759
• Accuracy Unlabeled: nan
• Accuracy Flat-road: 0.9069
• Accuracy Flat-sidewalk: 0.9471
• Accuracy Flat-crosswalk: 0.5043
• Accuracy Flat-cyclinglane: 0.8684
• Accuracy Flat-parkingdriveway: 0.5057
• Accuracy Flat-railtrack: 0.0
• Accuracy Flat-curb: 0.7351
• Accuracy Human-person: 0.8662
• Accuracy Human-rider: 0.2599
• Accuracy Vehicle-car: 0.9494
• Accuracy Vehicle-truck: 0.1607
• Accuracy Vehicle-bus: 0.0044
• Accuracy Vehicle-tramtrain: 0.1992
• Accuracy Vehicle-motorcycle: 0.0
• Accuracy Vehicle-bicycle: 0.7913
• Accuracy Vehicle-caravan: 0.4628
• Accuracy Vehicle-cartrailer: 0.0106
• Accuracy Construction-building: 0.9117
• Accuracy Construction-door: 0.2679
• Accuracy Construction-wall: 0.6351
• Accuracy Construction-fenceguardrail: 0.5893
• Accuracy Construction-bridge: 0.5639
• Accuracy Construction-tunnel: nan
• Accuracy Construction-stairs: 0.4246
• Accuracy Object-pole: 0.6323
• Accuracy Object-trafficsign: 0.4266
• Accuracy Object-trafficlight: 0.2431
• Accuracy Nature-vegetation: 0.9540
• Accuracy Nature-terrain: 0.8819
• Accuracy Sky: 0.9827
• Accuracy Void-ground: 0.0045
• Accuracy Void-dynamic: 0.2006
• Accuracy Void-static: 0.5328
• Accuracy Void-unclear: 0.0
• Iou Unlabeled: 0.0
• Iou Flat-road: 0.7947
• Iou Flat-sidewalk: 0.8656
• Iou Flat-crosswalk: 0.4529
• Iou Flat-cyclinglane: 0.6876
• Iou Flat-parkingdriveway: 0.4461
• Iou Flat-railtrack: 0.0
• Iou Flat-curb: 0.5989
• Iou Human-person: 0.6127
• Iou Human-rider: 0.2346
• Iou Vehicle-car: 0.8877
• Iou Vehicle-truck: 0.0662
• Iou Vehicle-bus: 0.0044
• Iou Vehicle-tramtrain: 0.1985
• Iou Vehicle-motorcycle: 0.0
• Iou Vehicle-bicycle: 0.5765
• Iou Vehicle-caravan: 0.1495
• Iou Vehicle-cartrailer: 0.0106
• Iou Construction-building: 0.8060
• Iou Construction-door: 0.2190
• Iou Construction-wall: 0.5015
• Iou Construction-fenceguardrail: 0.4923
• Iou Construction-bridge: 0.3467
• Iou Construction-tunnel: nan
• Iou Construction-stairs: 0.3908
• Iou Object-pole: 0.4693
• Iou Object-trafficsign: 0.3698
• Iou Object-trafficlight: 0.2052
• Iou Nature-vegetation: 0.8832
• Iou Nature-terrain: 0.7906
• Iou Sky: 0.9519
• Iou Void-ground: 0.0038
• Iou Void-dynamic: 0.1774
• Iou Void-static: 0.3885
• Iou Void-unclear: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Mean Iou	Mean Accuracy	Overall Accuracy	Accuracy Unlabeled	Accuracy Flat-road	Accuracy Flat-sidewalk	Accuracy Flat-crosswalk	Accuracy Flat-cyclinglane	Accuracy Flat-parkingdriveway	Accuracy Flat-railtrack	Accuracy Flat-curb	Accuracy Human-person	Accuracy Human-rider	Accuracy Vehicle-car	Accuracy Vehicle-truck	Accuracy Vehicle-bus	Accuracy Vehicle-tramtrain	Accuracy Vehicle-motorcycle	Accuracy Vehicle-bicycle	Accuracy Vehicle-caravan	Accuracy Vehicle-cartrailer	Accuracy Construction-building	Accuracy Construction-door	Accuracy Construction-wall	Accuracy Construction-fenceguardrail	Accuracy Construction-bridge	Accuracy Construction-tunnel	Accuracy Construction-stairs	Accuracy Object-pole	Accuracy Object-trafficsign	Accuracy Object-trafficlight	Accuracy Nature-vegetation	Accuracy Nature-terrain	Accuracy Sky	Accuracy Void-ground	Accuracy Void-dynamic	Accuracy Void-static	Accuracy Void-unclear	Iou Unlabeled	Iou Flat-road	Iou Flat-sidewalk	Iou Flat-crosswalk	Iou Flat-cyclinglane	Iou Flat-parkingdriveway	Iou Flat-railtrack	Iou Flat-curb	Iou Human-person	Iou Human-rider	Iou Vehicle-car	Iou Vehicle-truck	Iou Vehicle-bus	Iou Vehicle-tramtrain	Iou Vehicle-motorcycle	Iou Vehicle-bicycle	Iou Vehicle-caravan	Iou Vehicle-cartrailer	Iou Construction-building	Iou Construction-door	Iou Construction-wall	Iou Construction-fenceguardrail	Iou Construction-bridge	Iou Construction-tunnel	Iou Construction-stairs	Iou Object-pole	Iou Object-trafficsign	Iou Object-trafficlight	Iou Nature-vegetation	Iou Nature-terrain	Iou Sky	Iou Void-ground	Iou Void-dynamic	Iou Void-static	Iou Void-unclear
0.7275	2.5	500	0.5765	0.3050	0.3654	0.8441	nan	0.9561	0.9153	0.3719	0.7164	0.4360	0.0	0.3475	0.8270	0.0	0.9318	0.0	0.0	0.0	0.0	0.6843	0.0	0.0	0.9160	0.0667	0.3893	0.6512	0.0	nan	0.0	0.5447	0.0525	0.0	0.9581	0.8185	0.9737	0.0	0.0262	0.4752	0.0	nan	0.7208	0.8407	0.3582	0.6393	0.3693	0.0	0.2705	0.5291	0.0	0.8548	0.0	0.0	0.0	0.0	0.5158	0.0	0.0	0.7684	0.0638	0.3606	0.4620	0.0	nan	0.0	0.3805	0.0522	0.0	0.8572	0.7657	0.9225	0.0	0.0256	0.3078	0.0
0.3654	5.0	1000	0.5265	0.3531	0.4276	0.8622	nan	0.9116	0.9476	0.4986	0.8194	0.4632	0.0	0.5613	0.8672	0.1407	0.9399	0.2129	0.0	0.0	0.0	0.7640	0.0	0.0	0.8915	0.1468	0.5813	0.5719	0.0	nan	0.3336	0.5583	0.4068	0.0	0.9470	0.8512	0.9780	0.0004	0.1935	0.5228	0.0	nan	0.7904	0.8491	0.4455	0.6935	0.4013	0.0	0.4607	0.5435	0.1227	0.8663	0.1019	0.0	0.0	0.0	0.5464	0.0	0.0	0.7740	0.1198	0.4746	0.4587	0.0	nan	0.2403	0.4036	0.2949	0.0	0.8663	0.7614	0.9334	0.0003	0.1492	0.3544	0.0
0.2359	7.5	1500	0.5790	0.3584	0.4296	0.8649	nan	0.8646	0.9501	0.4466	0.8506	0.5513	0.0	0.7099	0.8317	0.2099	0.9442	0.2546	0.0	0.0	0.0	0.7862	0.0087	0.0	0.9079	0.1046	0.6479	0.5239	0.0	nan	0.1543	0.5674	0.3864	0.0	0.9461	0.8753	0.9776	0.0000	0.1974	0.4793	0.0	nan	0.7917	0.8414	0.4214	0.7012	0.4494	0.0	0.5333	0.5888	0.1897	0.8663	0.1009	0.0	0.0	0.0	0.5427	0.0042	0.0	0.7817	0.0945	0.4696	0.4312	0.0	nan	0.1477	0.4314	0.3152	0.0	0.8749	0.7719	0.9409	0.0000	0.1780	0.3593	0.0
0.1708	10.0	2000	0.6066	0.3684	0.4479	0.8666	nan	0.8819	0.9466	0.5609	0.8324	0.4835	0.0	0.7200	0.8575	0.1404	0.9422	0.2656	0.0	0.0590	0.0	0.7505	0.2619	0.0	0.8906	0.2203	0.6425	0.5323	0.0	nan	0.3455	0.5923	0.4085	0.0	0.9552	0.8844	0.9791	0.0024	0.0951	0.5293	0.0	nan	0.7930	0.8537	0.4562	0.6519	0.4311	0.0	0.5478	0.5960	0.1301	0.8728	0.1051	0.0	0.0590	0.0	0.5390	0.0916	0.0	0.7864	0.1779	0.4949	0.4467	0.0	nan	0.3107	0.4336	0.3289	0.0	0.8757	0.7694	0.9416	0.0019	0.0893	0.3741	0.0
0.1326	12.5	2500	0.5934	0.3969	0.4877	0.8753	nan	0.9227	0.9490	0.4762	0.8499	0.5255	0.0	0.6941	0.8115	0.3960	0.9430	0.3828	0.0266	0.0998	0.0	0.7963	0.6565	0.0003	0.8988	0.2880	0.6352	0.5442	0.2746	nan	0.3048	0.6133	0.4269	0.0	0.9483	0.9064	0.9838	0.0025	0.2297	0.5070	0.0	nan	0.8032	0.8631	0.4521	0.7678	0.4440	0.0	0.5593	0.6096	0.2994	0.8777	0.1308	0.0266	0.0998	0.0	0.5663	0.2016	0.0003	0.7931	0.1957	0.4838	0.4475	0.2094	nan	0.2849	0.4530	0.3424	0.0	0.8798	0.7777	0.9441	0.0023	0.1979	0.3857	0.0
0.1116	15.0	3000	0.6267	0.3978	0.4820	0.8734	nan	0.9155	0.9431	0.5145	0.8423	0.4973	0.0	0.7284	0.8513	0.3146	0.9492	0.0753	0.0	0.2565	0.0	0.7845	0.3853	0.0220	0.8998	0.2497	0.6306	0.5571	0.3186	nan	0.4912	0.5902	0.4795	0.0	0.9501	0.9004	0.9843	0.0038	0.2017	0.5677	0.0	nan	0.8037	0.8591	0.4665	0.7147	0.4311	0.0	0.5698	0.5996	0.2693	0.8800	0.0366	0.0	0.2554	0.0	0.5526	0.1200	0.0216	0.7935	0.1949	0.4853	0.4704	0.2116	nan	0.3980	0.4450	0.3691	0.0	0.8815	0.7857	0.9439	0.0031	0.1751	0.3899	0.0
0.098	17.5	3500	0.6334	0.3922	0.5006	0.8729	nan	0.8961	0.9419	0.5747	0.8862	0.4977	0.0	0.7428	0.8491	0.3477	0.9464	0.0952	0.0	0.2937	0.0	0.7908	0.7738	0.0	0.8934	0.2479	0.6445	0.6108	0.4273	nan	0.4435	0.6190	0.4308	0.0015	0.9486	0.9026	0.9818	0.0099	0.2216	0.4994	0.0	0.0	0.7961	0.8651	0.5005	0.6765	0.4413	0.0	0.5751	0.6176	0.2944	0.8811	0.0373	0.0	0.2919	0.0	0.5578	0.2307	0.0	0.7961	0.1835	0.4901	0.4814	0.2506	nan	0.3771	0.4560	0.3562	0.0015	0.8810	0.7806	0.9474	0.0077	0.1823	0.3784	0.0
0.0894	20.0	4000	0.6973	0.3988	0.4923	0.8722	nan	0.8952	0.9456	0.5309	0.8357	0.4777	0.0	0.7630	0.8291	0.2785	0.9467	0.1712	0.0047	0.1377	0.0	0.7854	0.8237	0.0	0.9282	0.1899	0.5904	0.6020	0.4761	nan	0.3323	0.6192	0.4047	0.1381	0.9522	0.8851	0.9767	0.0050	0.1916	0.5277	0.0	nan	0.8062	0.8543	0.4692	0.6675	0.4204	0.0	0.5710	0.6083	0.2366	0.8855	0.0677	0.0046	0.1374	0.0	0.5492	0.2453	0.0	0.7996	0.1571	0.4887	0.4847	0.2656	nan	0.3148	0.4670	0.3491	0.1264	0.8815	0.7829	0.9496	0.0043	0.1718	0.3936	0.0
0.0819	22.5	4500	0.6867	0.4098	0.5001	0.8778	nan	0.9344	0.9410	0.5690	0.8783	0.4856	0.0	0.7065	0.8495	0.2085	0.9415	0.1530	0.0018	0.2354	0.0	0.7829	0.7796	0.0	0.9044	0.2261	0.6171	0.6045	0.4780	nan	0.4156	0.6265	0.4288	0.1457	0.9563	0.8877	0.9804	0.0064	0.2136	0.5447	0.0	nan	0.8016	0.8702	0.4902	0.7597	0.4279	0.0	0.5780	0.6123	0.1998	0.8889	0.0577	0.0018	0.2348	0.0	0.5898	0.2436	0.0	0.7992	0.1842	0.4829	0.4918	0.2855	nan	0.3732	0.4658	0.3650	0.1297	0.8823	0.7837	0.9500	0.0053	0.1841	0.3839	0.0
0.0767	25.0	5000	0.7377	0.4096	0.5109	0.8720	nan	0.8599	0.9464	0.5724	0.9354	0.4838	0.0	0.7392	0.8475	0.2679	0.9530	0.2438	0.0	0.2405	0.0	0.7879	0.8364	0.0	0.9155	0.2107	0.5924	0.5901	0.5525	nan	0.3980	0.6229	0.4648	0.2165	0.9550	0.8865	0.9823	0.0047	0.1970	0.5557	0.0	nan	0.7881	0.8643	0.5042	0.6317	0.4280	0.0	0.5817	0.6075	0.2397	0.8857	0.1052	0.0	0.2384	0.0	0.5664	0.2501	0.0	0.8056	0.1813	0.4878	0.4863	0.2871	nan	0.3652	0.4725	0.3883	0.1660	0.8804	0.7905	0.9503	0.0040	0.1704	0.3891	0.0
0.0725	27.5	5500	0.7085	0.3977	0.5056	0.8782	nan	0.9177	0.9482	0.4916	0.8966	0.4989	0.0	0.7119	0.8469	0.2483	0.9512	0.2387	0.0440	0.1287	0.0	0.7947	0.8184	0.0	0.9152	0.2257	0.6472	0.5963	0.5426	nan	0.3951	0.6422	0.4369	0.2195	0.9499	0.8824	0.9821	0.0036	0.1824	0.5266	0.0	0.0	0.8109	0.8638	0.4498	0.7314	0.4437	0.0	0.5797	0.6047	0.2215	0.8861	0.0855	0.0430	0.1284	0.0	0.5657	0.2395	0.0	0.8058	0.1939	0.5113	0.4913	0.2943	nan	0.3732	0.4773	0.3770	0.1643	0.8836	0.7864	0.9509	0.0029	0.1639	0.3905	0.0
0.0685	30.0	6000	0.7388	0.4115	0.5051	0.8738	nan	0.9135	0.9420	0.5290	0.8405	0.4909	0.0	0.7408	0.8566	0.3161	0.9461	0.1138	0.0003	0.1616	0.0	0.8061	0.7486	0.0001	0.9074	0.2986	0.6418	0.5669	0.4769	nan	0.4607	0.6454	0.4717	0.2320	0.9531	0.8849	0.9802	0.0037	0.1983	0.5417	0.0	nan	0.7911	0.8647	0.4671	0.6651	0.4361	0.0	0.5848	0.6127	0.2642	0.8885	0.0453	0.0003	0.1613	0.0	0.5455	0.2421	0.0001	0.8022	0.2382	0.4975	0.4741	0.3279	nan	0.4050	0.4789	0.3937	0.1921	0.8825	0.7873	0.9516	0.0032	0.1809	0.3970	0.0
0.0654	32.5	6500	0.7246	0.4128	0.5034	0.8789	nan	0.9247	0.9424	0.5865	0.8579	0.5105	0.0	0.7409	0.8799	0.2449	0.9462	0.0922	0.0	0.1728	0.0	0.7762	0.7085	0.0	0.9151	0.2459	0.6278	0.6088	0.5426	nan	0.4260	0.6444	0.4471	0.2230	0.9530	0.8839	0.9833	0.0040	0.1978	0.5251	0.0	nan	0.8010	0.8705	0.5132	0.7193	0.4466	0.0	0.5906	0.5971	0.2204	0.8884	0.0419	0.0	0.1724	0.0	0.5623	0.2184	0.0	0.8044	0.2015	0.5037	0.4964	0.3206	nan	0.4032	0.4828	0.3859	0.1802	0.8828	0.7909	0.9505	0.0033	0.1792	0.3959	0.0
0.0629	35.0	7000	0.7655	0.4168	0.5105	0.8741	nan	0.8961	0.9470	0.5214	0.8906	0.4982	0.0	0.7542	0.8631	0.2754	0.9512	0.1882	0.0015	0.3457	0.0	0.7778	0.6418	0.0144	0.8908	0.2816	0.6612	0.5910	0.5330	nan	0.4434	0.6305	0.4273	0.2421	0.9516	0.8805	0.9821	0.0036	0.2172	0.5444	0.0	nan	0.7981	0.8672	0.4665	0.6765	0.4364	0.0	0.5934	0.6114	0.2489	0.8877	0.0831	0.0013	0.3436	0.0	0.5668	0.2017	0.0140	0.7928	0.2283	0.4708	0.4904	0.3458	nan	0.4011	0.4722	0.3699	0.1843	0.8836	0.7898	0.9516	0.0030	0.1851	0.3897	0.0
0.0607	37.5	7500	0.7668	0.4180	0.5139	0.8751	nan	0.8948	0.9480	0.5612	0.8579	0.4903	0.0	0.7432	0.8676	0.2619	0.9495	0.1718	0.0165	0.3359	0.0010	0.7738	0.7077	0.0304	0.9104	0.2826	0.6353	0.6045	0.5609	nan	0.4406	0.6293	0.4355	0.2376	0.9511	0.8940	0.9818	0.0033	0.2277	0.5530	0.0	nan	0.7933	0.8676	0.4914	0.6562	0.4327	0.0	0.5956	0.6059	0.2348	0.8875	0.0739	0.0161	0.3343	0.0010	0.5696	0.2086	0.0295	0.8084	0.2268	0.5014	0.4962	0.3297	nan	0.3948	0.4702	0.3754	0.1918	0.8836	0.7857	0.9519	0.0029	0.1886	0.3900	0.0
0.0582	40.0	8000	0.7562	0.4049	0.5074	0.8780	nan	0.9204	0.9463	0.5052	0.8643	0.5082	0.0	0.7360	0.8650	0.2462	0.9501	0.1893	0.0024	0.2403	0.0	0.7814	0.6631	0.0242	0.9169	0.2821	0.6356	0.5984	0.5609	nan	0.4218	0.6299	0.4414	0.2421	0.9504	0.8835	0.9796	0.0043	0.2138	0.5395	0.0	0.0	0.8004	0.8683	0.4553	0.7073	0.4478	0.0	0.6007	0.6132	0.2291	0.8882	0.0839	0.0024	0.2391	0.0	0.5777	0.2020	0.0237	0.8060	0.2276	0.5058	0.4990	0.3415	nan	0.3905	0.4699	0.3815	0.1952	0.8837	0.7908	0.9522	0.0036	0.1849	0.3941	0.0
0.0565	42.5	8500	0.7834	0.4004	0.5024	0.8762	nan	0.8960	0.9474	0.5430	0.8894	0.4937	0.0	0.7492	0.8696	0.2727	0.9482	0.1505	0.0006	0.1868	0.0	0.7945	0.5042	0.0051	0.9155	0.2834	0.6369	0.5958	0.5811	nan	0.4136	0.6419	0.4457	0.2481	0.9510	0.8887	0.9822	0.0041	0.2054	0.5358	0.0	0.0	0.7955	0.8676	0.4613	0.6877	0.4390	0.0	0.6023	0.6092	0.2503	0.8879	0.0626	0.0006	0.1860	0.0	0.5802	0.1628	0.0051	0.8061	0.2307	0.5003	0.4961	0.3290	nan	0.3820	0.4752	0.3819	0.2070	0.8838	0.7914	0.9520	0.0034	0.1813	0.3944	0.0
0.0562	45.0	9000	0.7812	0.4015	0.5008	0.8772	nan	0.9064	0.9466	0.5058	0.8872	0.5059	0.0	0.7482	0.8642	0.2957	0.9489	0.1494	0.0059	0.1674	0.0003	0.8079	0.4484	0.0122	0.9134	0.2785	0.6303	0.6007	0.5723	nan	0.4337	0.6286	0.4291	0.2541	0.9521	0.8902	0.9821	0.0054	0.2097	0.5444	0.0	0.0	0.7954	0.8685	0.4597	0.7046	0.4471	0.0	0.6024	0.6174	0.2611	0.8881	0.0630	0.0057	0.1667	0.0003	0.5844	0.1441	0.0120	0.8059	0.2279	0.5021	0.4963	0.3439	nan	0.3981	0.4694	0.3714	0.2134	0.8841	0.7883	0.9522	0.0044	0.1836	0.3913	0.0
0.0547	47.5	9500	0.7899	0.3997	0.4971	0.8759	nan	0.9053	0.9472	0.4999	0.8752	0.5002	0.0	0.7334	0.8557	0.2947	0.9505	0.1326	0.0	0.1843	0.0	0.8065	0.3995	0.0184	0.9146	0.2650	0.6301	0.6056	0.5749	nan	0.4294	0.6299	0.4450	0.2461	0.9515	0.8854	0.9825	0.0044	0.2045	0.5311	0.0000	0.0	0.7939	0.8655	0.4530	0.6865	0.4427	0.0	0.5983	0.6206	0.2592	0.8881	0.0597	0.0	0.1837	0.0	0.5769	0.1272	0.0183	0.8055	0.2174	0.5004	0.4960	0.3457	nan	0.3926	0.4724	0.3800	0.2072	0.8841	0.7912	0.9522	0.0037	0.1789	0.3895	0.0000
0.0543	50.0	10000	0.7826	0.3995	0.4977	0.8759	nan	0.9069	0.9471	0.5043	0.8684	0.5057	0.0	0.7351	0.8662	0.2599	0.9494	0.1607	0.0044	0.1992	0.0	0.7913	0.4628	0.0106	0.9117	0.2679	0.6351	0.5893	0.5639	nan	0.4246	0.6323	0.4266	0.2431	0.9540	0.8819	0.9827	0.0045	0.2006	0.5328	0.0	0.0	0.7947	0.8656	0.4529	0.6876	0.4461	0.0	0.5989	0.6127	0.2346	0.8877	0.0662	0.0044	0.1985	0.0	0.5765	0.1495	0.0106	0.8060	0.2190	0.5015	0.4923	0.3467	nan	0.3908	0.4693	0.3698	0.2052	0.8832	0.7906	0.9519	0.0038	0.1774	0.3885	0.0

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.0
• Datasets 2.14.5
• Tokenizers 0.14.1