pirst nd foremostD s would like to express my speil ppreition nd thnks to my thesis dviserD profF tosine eruiD you hve een gret mentor for meF s would like to thnk you for enourging my reserhD for lending support when s needed it nd for llowing me to grow s young sientistF s would like to thnk drF wthis yrtner from eirus hefense nd peD prneD your e'etive feedk nd insightful ides hve helped me drive my reserh forE wrdF our tive interest in my work nd your enthusism hve een very vlule to meF s would like to thnk the president of my orl defense ommittee nd former ollege teherD profF hniel hrieD for enling my (rst ontt with sxse nd the eyin temF s m forever grtefulF hnk you3 s would like to thnk my reding ommittee memersD profF elfred frukstein nd profF fExgu o for their timeD interest nd helpful ommentsF s would lso like to thnk the other three memers of my orl defense ommitteeD drF tques flnElonD profF vier hesomes nd profF wihel hmitt for their time nd interesting questionsF s would like to thnk my entire orl defense ommittee for mking my defense n enjoyle momentF s grtefully knowledge the funding soure tht mde my hFhF work possileF s ws funded y eirus hefense nd peD prneD

Introduction 11

IFI wotivtion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F II IFP ghllenges F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IQ IFQ epproh nd proposed methods F F F F F F F F F F F F F F F F F F F F IT IFQFI roilities in imge G video nlysis F F F F F F F F F F F F F IT IFQFP epplition to ojet detetion nd trking in videos F F F F IV IFR hesis orgniztion nd ontriutions F F F F F F F F F F F F F F F F F IW IFS ulitions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PH State of the art 21 PFI yjet detetion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PI PFIFI yjet detetion in stti imges F F F F F F F F F F F F F F F F PP PFIFP yjet detetion in dynmi imges F F F F F F F F F F F F F F PR PFIFQ e note on person detetion F F F F F F F F F F F F F F F F F F F PU PFP yjet trking F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PU PFPFI ht ssoition sed pprohes F F F F F F F F F F F F F F F PV PFPFP pinite et ttistis sed pprohes F F F F F F F F F F F F F PW
Using Point processes for object detection 33 QFI pundmentls F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QQ QFP rmeter estimtion F F F F F F F F F F F F F F F F F F F F F F F F F F F QV QFPFI wximum likelihood estimtion using wgwg tehniques F F F RH QFPFP ixpettion wximiztionElike lgorithms

F F F F F F F F F F F RI QFQ yptimiztion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F RR QFQFI he reversile jump wgwg smpler F F F F F F F F F F F F F F RS QFQFP wultiple irth nd deth lgorithm F F F F F F F F F F F F F F F SV QFQFQ tumpEdi'usion proesses F F F F F F F F F F F F F F F F F F F F F SV QFQFR imulted nneling F F F F F F F F F F F F F F F F F F F F F F F TH QFR gonlusions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F TP
Spatial marked point process model for boat extraction in harbors 63

RFI wodel F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F TR RFIFI ixternl energy term F F F F F F F F F F F F F F F F F F F F F F F TS RFIFP snternl energy term F F F F F F F F F F F F F F F F F F F F F F F TU RFIFQ otl energy term F F F F F F F F F F F F F F F F F F F F F F F F UI RFP rmeter estimtion F F F F F F F F F F F F F F F F F F F F F F F F F F F UI RFPFI hetermining the weights γ cnt nd γ al F F F F F F F F F F F F F F UQ RFPFP hetermining the weight γ c F F F F F F F F F F F F F F F F F F F F UQ RFPFQ hetermining the threshold d 0 F F F F F F F F F F F F F F F F F F UR RFQ yptimiztion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F UR iv Contents RFQFI erturtion kernels used for ojet detetion F F F F F F F F F US RFQFP i0ient implementtion of twgwg E multiple ores F F F F UU RFQFQ ter G vnd disrimintion F F F F F F F F F F F F F F F F F F VH RFR esults F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VQ RFRFI hetetion ury of the proposed model F F F F F F F F F F F VQ RFRFP gomputtionl e0ieny F F F F F F F F F F F F F F F F F F F F F VT RFS gonlusions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VU
5 From object detection to object tracking using a spatio-temporal marked point process model 89

SFI wodel F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F WH SFIFI snternl energy term F F F F F F F F F F F F F F F F F F F F F F F WP SFIFP ixternl energy term F F F F F F F F F F F F F F F F F F F F F F F WS SFIFQ otl energy term F F F F F F F F F F F F F F F F F F F F F F F F IHP SFP rmeter estimtion F F F F F F F F F F F F F F F F F F F F F F F F F F F IHP SFPFI viner progrmming F F F F F F F F F F F F F F F F F F F F F F F IHP SFPFP eight estimtion of individul energy terms s liner proE grmming prolem F F F F F F F F F F F F F F F F F F F F F F F F IHR SFQ yptimiztion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IHT SFQFI twgwg in Ph C F F F F F F F F F F F F F F F F F F F F F F IHU SFQFP roposition kernels used for ojet trking F F F F F F F F F F IHW SFQFQ gonsistent leling F F F F F F F F F F F F F F F F F F F F F F F F IIP SFQFR sntegrting ulmn like moves in twgwg F F F F F F F F F F IIT SFQFS i0ient implementtion of twgwg in Ph C E multiple ores F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IIW SFR esults F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPI
SFRFI rking results on syntheti enhmrks used y the iologE il proessing ommunity F F F F F F F F F F F F F F F F F F F F IPP SFRFP rking results on rel iologil dt F F F F F F F F F F F F F IPS SFRFQ rking results on simulted low temporl frequeny stellite dt

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPT SFRFR rking results on simulted high temporl frequeny stelE lite dt F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IQP SFRFS gomputtionl e0ieny F F F F F F F F F F F F F F F F F F F F F IQV SFS gonlusions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRS
6 Conclusions and perspectives 147

TFI hesis ontriutions F F F F F F F F F F F F F F F F F F F F F F F F F F F IRU TFP edvntges of the proposed methods F F F F F F F F F F F F F F F F F F IRV TFQ hrwks of the proposed methods F F F F F F F F F F F F F F F F F F IRW TFR erspetives F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRW TFS gonluding remrks F F F F F F F F F F F F F F F F F F F F F F F F F F F ISH Contents v A Appendix -Scientic activity eFI tournl ppers F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F ISI eFP gonferene ppers F F F F F F F F F F F F F F F F F F F F F F F F F F F F ISI eFQ snvited tlks F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F ISI eFR snterntionl ummer hool F F F F F F F F F F F F F F F F F F F F F F ISP B Introduction fFI wotivtion F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F ISQ fFP ves dé(s F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F ISS fFQ wéthodes et pprohes proposées F F F F F F F F F F F F F F F F F F F F ISW
fFQFI roilités dns l9nlyse des imges G du vidéo F F F F F F F ISW fFQFP epplition à l détetion et suivi des ojets dns des vidéos ITI fFR yrgnistion de thèse et ontriutions

F F F F F F F F F F F F F F F F F ITP fFS ulitions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F ITQ C Résumé étoé D Conclusions et perspectives hFI gontriutions de l thèse F F F F F F F F F F F F F F F F F F F F F F F F IUI hFP edvntges de méthode proposée F F F F F F F F F F F F F F F F F F F F IUQ hFQ snonvénients des méthodes proposées F F F F F F F F F F F F F F F F F IUQ hFR erspetives F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IUR hFS gonlusions F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IUS

Bibliography

List of Tables RFI gomprison of hrteristis of wter nd shdow omponents F F F VI RFP untittive nlysis of the proposed ot extrtion lgorithm for pot 5 imgesF he detetion ury dereses s the informtion on the orienttion of the ots eomes less relileF his is prtiuE lrly true in pigure RFIH @ottomA where the orienttion of the ots in the urved re is prtilly nonEexistentF F F F F F F F F F F F F F VQ RFR untittive nlysis of the proposed ot extrtion lgorithm for leides imgesF hue to the very high resolution of these imgesD the ojets no longer pper to e tngentF husD the overll ury is very goodF

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VS
RFQ untittive nlysis of the omputtionl e0ieny for ot extrE tion on pot

5 imgesF F F F F F F F F F F F F F F F F F F F F F F F F F F VT
RFS untittive nlysis of the omputtionl e0ieny for ot extrE tion on leides imgesF F F F F F F F F F F F F F F F F F F F F F F F F F VU SFI untittive nlysis of the detetion nd trking results otined using the uiltEin wr trker within sy @de ghumont et lFD PHIPAD the ontinuous energy minimiztion lgorithm developed y wiln et lF PHIR nd the proposed method for the three syntheti iologil imge sequenesF he proposed lgorithm hs the highest similrity sores wFrFtF the ground truthD oth in terms of trks nd detetionsF he proposed lgorithm outperforms urrent stte of the rt methods y more thn 5%F F F F F F F F F F F F F F F F F F F F F F IPP SFP hi0ult senrios with two or more rossing or yEpssing trjetories in syntheti iologil imge sequenesF pirst rowX qround truth trE jetoriesF eond rowX rjetories otined using proposed method @ sxseAF he trjetories losely resemle the ground truth trE jetoriesF vels re orretly preserved during the rossings or yE pssingF hird rowX rjetories otined using wr @sy pluginAF he trjetories re highly frgmentedF he lels re swithed or ojets re lost during rossings or yEpssingF F F F F F F F F F F F F F IPQ SFQ ulity of the detetion using the ttistil model @topA nd the ulity model @ottomAF F F F F F F F F F F F F F F F F F F F F F F F F F IPR SFR @A ixmples when the deterministi leling outperforms the splitGmerge pprohF @A ixmples when the reverse is trueF F F F F IPS SFS gomprison etween the results otined using the uiltEin wr trker within sy @de ghumont et lFD PHIPA nd the proposed method for the rel sp imge sequene @fsset et lFD PHIRAF xote howeverD tht the output of the wr trker should not e tken s ground truth informtionF he visul ssessment of the results revels tht the trks otined using the proposed lgorithm re more onsistent nd less frgmented thn those otined using the wr plugEinF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPT SFT wo hrd ot detetion nd trking sesF yjet 1 lmost lends with the kground y the end of the sequene nd is hrd to distinE guish from noise @wvesAF he proposed method suessfully detets the ojetD due to the joint optimiztion over detetions nd trksD while ll other trkers fil to distinguish the ojet from the kE ground noiseF yjet 2 exhiits strong pperne hnges due to its inrese in speedF he til of this ojet is mistken for new ojet y ll trkers exept the proposed methodF F F F F F F F F F F F F F F IPV SFU untittive results for the two sequenes of rel stellite imgesF he proposed method hs the highest preision nd the seond highest rell sores for the (rst sequeneD s well s the highest preision nd rell sores for the seond sequeneF he proposed methods sueeds in trking more trgets in eh sequene thn the other methods @wA nd lso hs the est detetion rtes for oth sequenes @AF IPW SFV untittive results for the two sequenes of stellite sequenes of high temporl frequeny @30rzAF he proposed method hs exellent preision nd etter rell thn the ulmn (lter nd the ristogrm sed trkerF xote tht ll trkers miss one ot @imge on the rightAF ine the veloity of this ot is very smllD frme di'erening fils to identify the ojet s foregroundF husD the ojet is never deteted throughout the sequeneF F F F F F F F F F F F F F F F F F F F IQQ SFW gomputtion times of the proposed multiple trget trker wFrFtF the numer of ores @g9sAF he otined results revel tht the proE posed method sles very well with lrge numer of oresF F F F F F IRI

List of Figures

IFI yne exmple for eh indited hllenge of moving ojet detetion nd trking with moving stellite sensor @ eirus h8AF @A mll ojet size leds to limited pperne modelingF sndividul ojet reognition is hrdF idges etween the two ojets re lurredF @A hdows st y tll uildings nd lrger ojets suh s usesF hdE ows owed to tll uildings inrese the di0ulty of ojet detetion due to the lower ontrst etween the ojet nd the kgroundF hdows owed to uses nd truks move long with the ojet nd n e mistkenly identi(ed s trgetsF @A gmer motion ounts for lrge distortions etween onseutive frmesF @dA F F F F F F F F F IS PFI @A mple imges of irplnes in eril nd stellite imges nd the edge detetion results otined using gnnyEherihe edge detetorF smge y rveen et lF PHISY @A rolisti templte of n irplneF e sliding window pproh n e used to detet irplnes in eril imgesF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F PQ PFP rtEsed ojet detetion proedureF pirstD sliding window serh is used to extrt pth imgesF xextD prts re deteted vi sprse representtion of the pthesF rough voting is performed using o'sets of deteted prts of the trget ojet nd (nllyD the mxim re found in the rough imgeF hese mxim represent the possile trgetsF smge y okoy nd swski PHIS F F F F F F F F F F F F F PQ PFQ epperne of people in top view eril nd stellite imgesF F F F F PU QFI imultion of oisson point proessF @A romogeneous oisson point proessY @A snhomogeneous oisson point proessF F F F F F F QR QFP imultion of truss intertion proess in squre window @l = 10A with (xed β = 10 nd rdius r = 1F @A γ = 0.0Y @A γ = 0.5Y @A γ = 1.0F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QU QFQ ixmple of modi(tion of the prmters of n ellipse with lol perturtion kernelsY @leftA ottionY @middleA rnsltionY @rightA leF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SQ QFR gell independeneF veftX gell independene is not ensuredF he width of the ell is not lrge enough nd heneD perturtion of the ojet in ell c 1 depends on the ojet in ell c 3 Y ightX gell independene is ensuredF he width of the ell is lrge enough so tht ny perturtion of n ojet in ell c 1 would not depend on n ojet in ell c 3 F smge ourtesy of erdie PHIQF F F F F F F F F F F F SU QFS egulr prtition sheme on KD with dim K = 2 @leftA nd dim K = 3 @rightAF smge ourtesy of erdie PHIQF F F F F F F F F F F F F F F F F SU QFT @A en imge of otsY @A he lss of interest is estimted from the input imge @for exmpleD irth mp n e used to otin this initil estimtionAY @A he orresponding proilities q c,i D when regulr prtitioning sheme is ppliedY @dA he orresponding proE ilities q c,i when dtEdriven prtitioning sheme is ppliedF F F F F SU RFI veftX rtiulr se of hror where ll ots hve the sme oriE enttionF ightX qenerl se of hrors where ots hve di'erent orienttionsD sed on their position in the hrorF F F F F F F F F F F TR RFP @A gloseEup on n re with neighoring ots within the hrorY @A e typil exmple of ot where the pilothouse nd superstrutures pper s drk spots within the otY @A sn redX the exterior order F ρ (u) used to ompute the ontrst distne to the interior of the ellipseY @dA sn redX the exterior order F ρ (u) nd in greenX the interior order

I 1 (u)F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F TS
RFQ veftXisul representtion of the qulity funtion used to onstrut the externl term in eqution RFSF he more n ellipse from onE (gurtion x overlps n ojet in the imgeD the lower the vlue of the qulity funtion nd thusD the lower the externl energy termF sf n ellipse does not overlp with n ojet in the imgeD the qulity funtion returns high vlue whih in turn leds to higher vlue for the externl termD mking the ellipse less prole to e prt of the true on(gurtionF ightX isul representtion of the qulity funtion wFrFtF its rgumentF F F F F F F F F F F F F F F F F F F F F F F F TT RFR isul representtion of overlpping ellipses wFrFtF (xed overlpping rtio s = 0.5F he energy of the on(gurtion inreses s the mount of overlpping etween ojets inresesF F F F F F F F F F F F F F F F TU RFS isul representtion of the onepts of reference circles nd center pixels used to determine the lol orienttions of ojets in the seneF TW RFT orkE)ow of the lol orienttion detetion methodF opEleftX snitil imgeF opErightX isuliztion of the referene irlesF he righter the pixel vluesD the lrger the rdius of the referene irleF fottomE rightX genter pixel extrtion sed on the rdius of the referene irlesF fottomEleftX vines otined using the rough trnsformF he orienttion of these lines is used to determine the lol orienttion of the ojetsF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F UH RFU e visul representtion of the order width ρ for few seleted ots of di'erent sizesF @A he input imgeF @A ρ = 1F his is good vlue for very smll otsD howeverD it does not provide su0ient numer of pixels to ompute the relevnt sttistisF @A ρ = 2F his is the est vlue tht (ts oth smll nd lrge otsF wen nd vrine of the order pixels n e omputed for smll ots s wellF @dA ρ = 3F sn this seD the order signi(ntly overlps with neighoring ojets @oth ots nd doksA nd the ontrst vlue drops ordinglyF por lrge ojetsD the dditionl pixel in the order width does not led to signi(nt hnges in the order sttistisF F F F F F F F F F F F F F UP

RFV veftX e lrge ot split in two prts due to the spe prtitioningY widdleX he ojet is split t ell oundry leding to the detetion of two smller otsY ightX he ot is deteted twie if ellipses re llowed to ross the ell oundryF F F F F F F F F F F F F F F F F F F F UU RFW @AD @A smge of ots inside nd outside hror eirus h8Y @AD@dA terGvnd disrimintion results fter pplying threshold to the input imgeF righ flse lrm rte due to shdowsY @eAD@fA terGvnd disrimintion results using the lgorithm desried in elgorithm VF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VP RFIH @A pot 5 imges of ots in hror on the prenh ost gxiY @A rough lines otined for the extrtion of lol orienttion inE formtion for the otsY @A fot extrtion resultsF e totl of 501 @topAD 169 @seond rowAD 427 @third rowA nd 96 @ottomA ots re deteted respetivelyF F F F F F F F F F F F F F F F F F F F F F F F F F F F VR RFII @A leides imge of ots in hror in welourneD eustrli eirus h8Y @A rough lines otined for the extrtion of loE l orienttion informtion for the otsY @A fot extrtion resultsF e totl of 31 @topA nd 39 @ottomA ots re deteted respetivelyF VS SFI esults of frme di'erening for omputing the ojet evidene termF @A smge of ots in welourneD eustrli eirus h8Y @A prme di'erening of two onseutive frmesY @A prme di'erening results fter pplying wter mskY @dA prme di'erening results fter pE plying wter msk nd performing morphologil erosion nd losE ing opertions to smooth the oundries of the foreground regionsF F WP SFP wo exmples of trks over three frmes tht will e identilly rewrded y the dynmi energy termD s this term fvors ojets tht follow onstnt veloity modelD regrdless of the tul veloity of the ojetF he green rrow shows the diretion nd the mgnitude of the veloityF veftX he ojet depited hs slow veloity over the three frmesY ightX he ojet hs very high veloity over the three frmesF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F WR SFQ @A e typil exmple of otY @A sn redX the exterior order F ρ (u) nd in greenX the interior order I 1 (u) used to ompute the ontrst distne mesure for ojet detetion in ghpter RY @A sn redX the exterior order F ρ (u) used to ompute the ontrst distne to the interior of the ellipse for trking purposes in this hpterF F F F F F WT SFR yg urves for the syntheti iologil dt set with 3 di'erent winE dow sizesX 9 × 9 @redAD 11 × 11 @lueA nd 15 × 15 @greenAF he yg urves show tht window size of 9 × 9 gives the est resultsF F F F F IHH SFS veftX yntheti iologil imge ontining 32 trgetsF widdleX he output of M t GLRT F ightX reEdetetion resultsF F F F F F F F F F F F F IHH SFT veft topX tellite imge of oulon ontining 2 trgets @otsAF veft ottomX reEdetetion resultsF he flse lrms re used y wves hitting the shoreD resulting in high hnges etween onseutive frmesF ightX he output of M t GLRT F F F F F F F F F F F F F F F F F F IHH SFU he e'ets of di'erent omponents of the energy termsF he upper row shows on(gurtion with higher energy vlue for eh indiE vidul termF he ottom row shows on(gurtion with lower energy vlue for eh individul termF he drk spots denote trE get lotions t di'erent time frmesF hi'erent olors on the trgets represent di'erent lels ssigned to ehF F F F F F F F F F F F F F F F IHI SFV he verge normlized error || Ĉ-C||/||C|| with respet to the numE er of onstrintsF F F F F F F F F F F F F F F F F F F F F F F F F F F F F IHT SFW @A ixmple input imge from the welourne dt set @ eirus h8AY @A ixtrted wter re using the lgorithm presented in RY @A snitil irth mp otined using simple threshold on the input imge nd restrited to the wter re using the wter mskY @dA isuliztion of n event oneF he one rehes oth forwrds nd kwrds in timeF he event one is used to inrese the proility of detetion in the volume it in)uenesF F F F F F F F F F F F F F F F IHU SFIH he stte of the wrkov hin depends on the order in whih the ojets re dded to the on(gurtionF gse IX X = ((X ∪ u) ∪ v) ∪ w @(rst uD then v nd then w ws retedAF gse PX X = ((X∪u)∪w)∪v @(rst uD then w nd then v ws retedAF F F F F F F F F F F F F F F F F IIS SFII hetetion nd trking results otined on three syntheti iologil imge sequenes of 100 frmes eh @ sxseAF ih olor represents di'erent trkF yjets n pper nd dispper t ny lotion nd time instneF ih sequene hs di'erent level of qussin noise @leftX no noiseY middleX µ = 25D σ = 2.5Y rightX µ = 50D σ = 5.0AF IPP SFIP hetetion nd trking results on rel iologil sp sequene of 300 imges @y ourtesy of tF lmeroD sg sfieD w IRR gx snstitut gurie @fsset et lFD PHIRAA @ sxseAF he imge sequene shows ellD with the orresponding vesiles tht trnsport sustnes inside itF he gol is to detet nd trk these vesilesF he visul ssessment of the results revels their very good qulityF F IPT SFIQ hetetion nd trking results on two sequenes of rel stellite imE ges tken t di'erent ngles @ sxseAF ih olor represents di'erent trkF veftX rking results on the (rst imge sequene up to frme 10F ightX rking results up to frme 13 of the seond imge sequeneF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPW SFIR hisplement etween two onseutive frmesF he ojets exhiit lrge jumps from one frme to the nextF histortions re lso visile on the lnd reF

F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPW
SFIS @A smge displement etween two imges 20 frmes prt in the frelon sequene t 3rz temporl resolutionY @A hetetion reE sults of possile moving trgets otined using the motion detetor provided y eirus h8F he detetor n hndle geometri distorE tions nd mer movementsF he irles represent possile deteE tions with vrious proilities @yellow a lowD red a highAY @A he motion )ow in the entire sequeneF vighter res highlight lotions with intense motion throughout the sequeneF he proposed model minly identi(es the ojets whih re on lnd with few flse lrms round the ostF roweverD y tking into ount the temporl inE formtionD the high numer of flse lrms generted y the detetion preEproessing step is onsiderly reduedY @dA rgets deteted t frme 170 of the sequene using the proposed methodF he distint olors of the ojets represent di'erent trksY @eA rgets deteted t frme 170 using the extended ulmn (lter nd smootherF ih ojet is ssigned numer to represent di'erent trksY @fA rgets deteted t frme 170 using the histogrmEsed trkerF ell ojets hve the sme olorF veling is not shown on this imgeF F F F F F F IQI SFIU smge displement etween two imges 20 frmes prt in the oulon sequene t 30rz temporl resolutionF he only visile displement is in the motion of the otsF xote howeverD tht the displement of the undeteted ot is not visile even fter 20 frmesFIQQ SFIV hetetion nd trking results on two sequenes of simulted stellite imges of oulonF he proposed method onsistently misses one ot @imge on the rightA euse of its very smll veloityF he imge sequenes re y ourtesy of eirus hefense 8 peD prneF F F F F IQQ SFIT @A hisplement etween two imges 20 frmes prt in sequene with high temporl frequenyF he imge sequene is y ourtesy of eirus hefense 8 peD prneF @A hetetion nd trking results for the frme 94F @A r0 density fter 500 frmesF @dA rking results otined y up in frme 94F F F F F F F F F F F F F F F F F F F IQR

List of Figures

SFIW hetetion nd trking results for the frme 94 of the pormul 1 seE queneF opX esults otined using the ulity modelF he model o'ers good detetion rte with zero flse lrmsF fottomX esults otined using the ttistil modelF he model o'ers etter deE tetion rte t the ost of n inresed flse lrms rteF he imge sequenes re y ourtesy of eirus hefense 8 peD prneF F F F F IQS SFPH hetetion nd trking results on sequene of simulted stellite imges of frelonF he tr0 density fter 500 n e oservedF opX he tr0 density otined using the ulity modelF fottomX he tr0 density otined using the ttistil model with lrger numer of deteted trgets t the ost of high flse lrm rteF he imge sequene is y ourtesy of eirus hefense 8 peD prneF F F IQT SFPI hetetion nd trking results on sequene of simulted stellite imges of frelonF he tr0 density fter 500 n e oservedF opX he tr0 density otined using the ulity modelF fottomX he tr0 density otined using the ttistil model with lrger numer of deteted trgets t the ost of high flse lrm rteF he imge sequene is y ourtesy of eirus hefense 8 peD prneF F F IQU SFPP inergy evolution with the numer of itertions for the stndrd twE gwg smpler @greenA nd the twgwg with ulmnElike moves @lueAF he e0ieny of the stndrd twgwg smpler is signi(E ntly inresed y inorporting the properties of sequentil (lters into the irth nd deth perturtion kernelF @AE@A esults on two enhmrk iologil sequenesY @AE@dA esults on the two high resE olution stellite imge sequenes with high temporl frequeny from the oulon dt setF F F F F F F F F F F F F F F F F F F F F F F F F F F F IQW SFPQ snrese in omputtionl e0ieny of the proposed multiple ojet trker wFrFtF the numer of ores @g9sAX @leftA for enhmrk iologil imge sequenesY @rightA for the frelon dt set with high temporl frequenyF F F F F F F F F F F F F F F F F F F F F F F F F F IRI SFPR inergy levels rehed wFrFtF time for di'erent smplersF F F F F F F F IRR SFPS he eptne rte wFrFtF to the temperture prmeter T ∈ [0, 100]D for ll the moves for the smplers presentedF e temperture prmE eter T ∈ [0, 15] o'er good proposl distriutions to smple fromF gonsequentlyD in our experiments we set the initil temperture to T = 15 to otin lne etween the eptne rte nd onverE gene speedF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IRR fFI n exemple pour hque dé( de l détetion d9ojet et le suivi à prtir d9un pteur stellitire donné @ eirus h8AF @A v petite tille de l9ojet onduit à une modélistion de l9pprene limitéeF v reonnissne d9un ojet individuel est di0ileF ves ords entre les deux ojets sont )ousF @A ves omres projetées pr de grnds immeules ugmentent l di0ulté de détetion d9ojets en rison du file ontrste entre les ojets et l9rrièreEplnF ves omres dues ux utous et mions se déplent ve l9ojet et peuvent être identi(ées pr erreur ommes des ilesF @A ve mouvement de l mér produit des grndes distorsions dns des trmes onséutivesF @dA ixemple de zone urine denseF 1.1 Motivation he lst dede hs een showse for symmetri wrfre with demonstrtions of highly orgnized ttks nd dvned ommunition nd plnning methods uydd nd lterD PHHTD olndD PHIHF es onsequeneD the ivilD militry nd eonomi seurity is thretened y wellEdoumented ssssintion ttemptsD hostgeEtking nd terrorist ttksF opulr methods suh s soil medi nlysis puhsD PHHW or eletroni evesdropping vnduD PHII re not su0ient to prevent or detet ongoing ttksF fut when these methods filD moile irorne nd speorne surveillne n help to detet riminl tivities efore or during their exeutionF rumns however re not the only ones who n ene(t from overhed surveillneF sn the lst dedeD hundreds of niml speies hve eome endngered y exesE sive hunting or glol wrmingF por exmpleD this ltter phenomenon led to n elerted loss of erti se ie during reent yers gomiso et lFD PHHVD troeve et lFD PHIP whih hs high impt on erti wildlifeF yverhed surveillne n id in ssessing the trends in undne of the erti speies pretwell et lFD PHIPD ltonov et lFD PHIQ without onerns out intrusive study methods or humn sfetyF tellites n e equipped with vriety of sensors suh s oustiD rdrD ultrE soni or imging sensorsF ih sensor hs dvntges nd drwks nd the hoie of whih type of sensor to tth to stellite depends on the spei( pplitions for whih tht stellite is uiltF tellite dt hs lredy een suessfully used for serh nd resue opertions vukowski nd ghronneuD PHHHD ting nd hnpingD PHIID nturl disster relief oigt et lFD PHHUD hell9equ et lFD PHIID fhngle

F F F F F F F F F F F F F F F F F F F F F F F F F F ISV
Chapter 1. Introduction nd hurhD PHIR or environmentl monitoring gett et lFD PHIID ingh nd lwrD PHIQF ehnologil dvnes in the stellite industry re now enling fr more urte nd relile imgery whih hs the potentil to revolutionize the 21Est entury surveillne nd monitoring prtiesF egrdless of the type of sensors mounted on the stelliteD nlyzing the quired dt is di0ult nd tedious jo for humn opertors hrterized y high levels of ftigue nd oredom qriD PHHU due to the lrge mount of informtion ville in the dtF epproprite lgorithms for utomti nd semiEutomti dt proE essing nd informtion retrievl n ssist the humn opertorF xeverthelessD for most pplitions it is hllenging to gurntee low error rte nd high on(dene of n lgorithm while lso providing ner relEtime performneF his thesis fouses on the nlysis of stti nd dynmi imges oming primrily from onEord very high resolution optil sensorsF sn prtiulrD we re ddressing two min prolemsX ojet detetion in stti imgesD respetively detetion nd trking of moving ojets in videosF yjet detetion nd respetively trking re two intermediry steps for utomti sene understndingF hese omplited ptterns of highElevel informtion n then e used to model nd detet suspiious ehviors nd outliers whih n e inditive of riminl ehviorsF smge nd video sed methods provide vlule informtion s mny properties of the deE teted ojets suh s positionD sizeD pperne or shpe n e diretly derived from the dtF st is importnt to note tht we use the term 9video9 for sequene of imges tken t reltively short time intervlsF he temporl frequeny of the videos n rnge from pproximtely 1 -2rz to 30rz or moreD depending on the type of sensor usedF por exmpleD single lowEorit stellite @suh s kytE1A n produe suEmeter resolution imgery nd highEde(nition video pturing up to 90Eseond video lips t 30 frmes per seondF roweverD lrge numer of lowEorit stellites re needed to over given re for long period of timeF sf this is the intentD then geosttionE ry stellite ould provide n lterntive to onstelltion of lowEorit stellitesF qeosttionry stellites re irthEsynhronousD mening tht they rotte with the irth @s opposed to lowEorit stellites whih re unEsynhronousA nd n over lrger re on groundF gonsequentlyD suh stellite would produe very lrge imges t lower temporl frequenyF his is why we onsider wide rnge of temporl frequenies for the videos onsidered in this thesisF tellite imge nlysis is not the only domin where humn opertors re overE loded y the mount of dt villeF wirosope imge nd video qulity hs experiened lrge inrese in reent yers nd the nlysis of mirosope dt is now ommonple in (elds suh s mediine or iologil reserhF he high quiE sition speed of the mers llows the rel time oservtion of dynmi ellulr nd suEellulr proessesF es suhD this pproh n generte vst mounts of dt tht hve to e storedD proessed nd nlyzedF eutomti nd semiEutomti methods for informtion retrievl re of gret importne to the humn opertor in this (eldF gonsequentlyD we lso test the models developed in this thesis on syntheti nd rel iologil imge sequenesF e re prtiulrly interested in the detetion 1.2. Challenges 13 nd trking of suEellulr strutures lled vesilesD whih re responsile for the proper funtioning of living ellF en inEdepth understnding of the dynmi nd geometril ptterns of the vesiles nd the ell in generlD n led to deeper iologil understnding nd more dequte medil tretmentsF gomplited geometril ptterns often require sttistil nlysisF epproprite stE tistil tools nd suitle mthemtil models re neessry to nlyze this dtF tohsti geometry is one of the res of mthemtil reserh whih seeks to provide suh methods nd modelsF he modern theory of stohsti geometry pioE neered y hFqF uendllD uF urikererg nd FiF wiles exmines rndom geometri ptterns of omplited distriutions toyn et lFD IWVUD toyn nd toynD IWWRD vn vieshout nd fddeleyD PHHPF ptil point proesses nd in prtiulr mrked point proesses hve lredy een suessfully pplied to ojet detetion prolems in very high resolution stellite imgery yrtnerD PHHRD errin et lFD PHHSD voste et lFD PHHSD hesmps et lFD PHHVD yrtner et lFD PHHVD hesomes et lFD PHIIF his thesis fouses on the use of sptil nd sptioEtemporl mrked point proess models for ojet detetion nd trking in vrious types of optil imgeryF he im is to exmine if mrked point proesses n produe ompetitive results for these tsksF he underlying theoretil foundtion of this thesis is sed on the existing litertureD ut severl novel ides nd pprohes re introdued in order to dpt nd improve existing methods with respet to ojet detetion ury nd to propose new multiple ojet trking pproh sed on mrked point proesses with good trdeo' etween trking performne nd runEtimeF 1.2 Challenges he use of remotely sensed stellite video to detet nd trk ojets is hllenging tskF hese hllenges re enountered throughout the proessing hinD from imge quisition to nlysisF eording to the ourrene time nd pleD they n e tegorized s followsX IF Image / video acquisition • Sensor noise represents devition from the optiml rdiometri vlues of the imge pixelsF hepending on the type of sensor usedD the noise n e modeled s either dditiveD multiplitive @spekleA or impulsive @sltE ndEpepperA qonzlez nd oodsD PHHVY • Small object size re onsequene of the ground smpling resolution of the sensors onsidered in this thesisF yjets re typilly few pixels in size @rnging etween 10 E 100 pixelsA whih mkes detetion nd reognition prtiulrly di0ult sine the mount of informtion ville on ojet pperne nd shpe is very limitedF por exmpleD urn senes re hrterized y lrge numer of trgets loted lose to eh other whih generlly results in two smllD neighoring ojets eing detet s one lrge trgetF ehile tr0 on usy min rod or ots in hrors re exmples of suh sesY

• Shadows n e st either y the trgets themselves or y the surE rounding ojetsF sn n urn environmentD shdows n e st y lrge vehiles suh s truks nd uses or y tll uildingsF es resultD some trgets ould e missed or shdows n e identi(ed s trgetsY

• Independent sensor and object motion dds dditionl hllenges to the detetion of ojetsF smge registrtion nd wrping itovà nd plusserD PHHQ re used to ompenste for the mer motionF henD moving ojets n e deteted y their reltive motion with respet to the sttionry kgroundY • Time requirements pose gret hllenge to ojet trking lgoE rithmsF sn most pplitionsD trdeEo' etween the qulity of the reE sults nd the proessing time hs to e mdeF his prolem eomes even more pprent in omplited senes with lrge numer of ojetsF he hllenges desried ove give )vor of why the tsk of ojet detetion in stti imgesD respetively the detetion nd trking of moving ojets in videos is very di0ultF his thesis is not intended to ddress ll the hllenges mentioned oveF eent developments in imge denoising ho et lFD PHIRD imge restortion ortill et lFD PHHQD imge delurring hng et lFD PHIQ or temporl (ltering wüller nd wüllerD PHIH provide stteEofEtheErt methods to del with most of the hllenges during quisitionD trnsfer nd initil proessing of the stellite dtF his thesis howeverD ddresses nerly ll the hllenges of imge G video nlysis @A @A @A @dA pigure IFIX yne exmple for eh indited hllenge of moving ojet detetion nd trking with moving stellite sensor @ eirus h8AF @A mll ojet size leds to limited pperne modelingF sndividul ojet reognition is hrdF idges etween the two ojets re lurredF @A hdows st y tll uildings nd lrger ojets suh s usesF hdows owed to tll uildings inrese the di0ulty of ojet detetion due to the lower ontrst etween the ojet nd the kgroundF hdows owed to uses nd truks move long with the ojet nd n e mistkenly identi(ed s trgetsF @A gmer motion ounts for lrge distortions etween onseutive frmesF @dA presented oveF he poor qulity of the dt is hndled impliitly through the noise resistne inorported into the lgorithmF e visul representtion of the hllenges whih rise during imge G video nlysis is presented in pigure fFIF pigure fFI @A shows two neighoring rs on the rodF wodeling the pperne of the vehiles is di0ult s there exists limited mount of informtionF purthermoreD during the overtking the rs re very lose to eh other whih mkes individul detetion prolemti s the oundries etween the vehiles eome lurredF pigure fFI @A illustrtes the dded di0ulty used y shdowsF he shdow st y tll uilding ould totlly olude n ojet pssing through tht reF hdows n lso e st y lrge trgets suh s truksF his n eome prolem espeilly when multiple lrge trgets re driving together in groupF he detetion nd trking lgorithm n misledingly interpret the entire group s single moving ojetF pigure fFI @A shows the di0ulty of deteting moving ojets in the presene of Chapter 1. Introduction mer motionF he olors in the imge show the mer displement etween two onseutive frmesF eny motion detetor tht does not tke the mer motion into ount would produe signi(nt numer of flse lrms sine stti ojets lso pper to move due to the mer displementF pigure fFI @dA shows n exmple of dense urn reF he sene ontins lose to 100 vehilesF ih vehile in the smll window hs een mnully leled with red ounding oxF he mnul leling is used s ground truth informtion to ssess the performne of the proposed utomti detetion nd trking lgorithmF e timeEe0ient lgorithm should signi(ntly derese the proessing time required ompred to humn opertorD while mintining high ury levelF everl pprohes hve een developed in the pst to meet the hllenges presented ove nd re presented in etion PF xeverthelessD there is lrge potentil to improve over the existing methods in terms of roustness nd proessing timeF 1.3 Approach and proposed methods he ojetive if this thesis is to study the use of geometri onstrints in ojet detetion nd trking methods through mrked point proessesF his pproh enles the inlusion of two types of geometri onstrintsX the shpe of the trget n e enfored @s we re deling mostly with mnEmde ojetsD their shpe n usully e e0iently pproximted using simple prmetri shpeA nd the geometri dispersion of the trgets n e onstrinedF oint proesses re lss within the generl domin of stohsti geometry nd hve lredy een suessfully pplied to ojet detetion prolems in stellite imgeryF sn the followingD short desription of why these mthemtil lss is of interest in imge G video proessing is presentedF 1.3.1 Probabilities in image / video analysis sn imge nlysisD mrked point proesses re nturl evolution of wrkov (elds s result of the signi(nt inrese in the sptil resolution of the imgesF he reder interested in detiled nlysis of this evolution n refer to the rd theses of érez PHHQ nd hesomes PHHRF 1.3.1.1 The stochastic model of an image en imge is olletion of pixelsF en imgeD denoted ID is funtion whih ssoites gry vlue to eh pixel

uD u ∈ K ⊆ Z 2 X I : K → R u → I(u).
X : x = {x 1 , • • • , x n(x) }D where x i , i = 1, n(x) is geometri ojet rnE domly pled in n imge nd n(x)
is the numer of ojetsF henD in fyesin frmeworkD n oservtion model is given for exmple yX

P(I(u) = y u |X = x) = 1(u ∈ x)P(I(u) = y u |u ∈ x) + 1(u / ∈ x)P(I(u) = y u |u / ∈ x).
@IFSA sn this exmplesD ll pixels u elonging to ny ojet of the on(gurtion form the silhouette of the on(gurtionF errin et lF PHHS proposed to use the rdiometri Chapter 1. Introduction vlues of the pixels to determine whether they elong to the on(gurtion or to the kgroundF es suhD the pixels elonging to the on(gurtion were modeled s qussin distriution with high men vlueD while the pixels elonging to the kground were modeled s qussin distriution with lower men vlueF his fyesin model enounters two di0ulties in rel imges fen rdj et lFD PHIHX IF he kground lss is in generl not homogeneous nd to ssume n qusE sin distriution for the entire kground is not relisti for our prolemF por exmpleD onsider n urn seneF sf we wnt to detet rs on the rodD it might e resonle to ssume tht the re surrounding single r @the rodA follows qussin distriutionD while the entire urn re does not @s it ontins shdowsD uildingsD treesD etFAY PF he likelihood of the pixels whih elong to the on(gurtion does not tke the morphologil properties of the ojets into ountF por exmpleD if we onsider n imge ontining rightD lrge retngle nd we re interested in deteting rightD smll ellipsesD the fyesin model will (t s mny ellipses s possile within the retngleD whih is not the desired resultF tepping outside the fyesin frmeworkD these limittion hve een surpssed y detetor pproh in whih the oservtion model is omposed of lol likelihoods for eh ojet tht is prt of the on(gurtionF his pproh hs enled the use of mrked point proesses for vrious ojet detetion tsksF hesmps et lF PHHV used mrked point proess of ellipses to ount the numer of )mingos in lrge olonies ontining thousnds of suh speimensF errin et lF PHHS developed mrked point proess of irles for treeErown extrtionF ris models ould lso identify the shdow st y the tree nd s suh determine the position of the un when the imge ws tkenF yrtner PHHR proposed mrked point proess of retngles for uilding detetionF elignment intertions etween the retngles were de(ned to reprodue the lyout of the uildingsF wore reentlyD fen rdj et lF PHIH introdued mrked point proess of ellipses to ount ots in hrorsF gonsequentlyD point proess models hve shown their potentil in ojet detetionF 1.3.2 Application to object detection and tracking in videos he im of this thesis is to study the possiility to use mrked point proess models for the detetion nd trking of moving ojets in video dtF he detetion nd trking of moving ojets plys n importnt role in video nlysis where it serves s neessry step in performing highElevel nlysisD for instne terrestril nd mritime tr0 monitoringF he prolem of ojet detetion in stti imges is lso disussed in this thesisD with strong emphsis on the prtiulrly di0ult prolem of ot extrtion in hrorsF he min dt used in this these re very high resolution stellite imges nd videos whih hve een supplied y eirus hefense nd peD prneF he dt sets vry 1.4. Thesis organization and contributions oth in size nd in the temporl frequeny nd rnge from shortD low temporl frequeny dt sets @14 frmes t 1 -2 rzA to longD high temporl frequeny videos @over 3000 frmes t 30 rzAF he ground truth informtion for these dt sets hs een onstruted y hnd nd vlidted fterwrd y n expertF eondry sets of dt oming from the iologil domin hve lso een nlyzed in this thesisF hese dt sets hve een provided y steur snstitute in risD prne nd gurie snstitute in risD prneF purthermoreD free softwre nmed sy developed y the untittive enlysis nit from steur snstitute in ris hs een used to utomtilly rete syntheti iologil imges nd videos with the ssoite ground truth to ssess the qulity of the models proposed in this thesisF his hpter overs relted work on ojet detetion nd trking in high resoluE tion remotely sensed imges pplied to similr dt sets nd fing similr hllenges s in this thesisF he fous of the literture review will e on stellite nd eril imgery nd the onsidered tsks will inlude oth detetion in stti imges s well s detetion nd trking in videosF he imge nd video dt onsidered in the literture under review re oming oth from lowEorit stellites @eFgF leidesD qeoiyeD kytA ut lso from es or irE plnes )ying t di'erent ltitudesF he mer ngle lso vries etween topEview vvigne et lFD PHIHD io et lFD PHIHD vuo et lFD PHIP nd olique view go et lFD PHIID gherghi nd heikhD PHIPD im nd ilrelwD PHIP for detetion nd wide re surveillne erer et lFD PHHTD eily et lFD PHIHD leemi nd hhD PHIQF eserh work on stellite videos is only in its infny sine the tehnologil dE vnes to pture video dt @whether low or high temporl frequeny dtA hve een very reentF roweverD top view high ltitude eril dt looks very similr to stellite dtD whih is why we inlude it in our literture reviewF e strt this hpter y reviewing ojet detetion methods in stti high resoluE tion stellite imges nd then turn our ttention to ojet detetion methods for video dt efore swithing to ojet trkingF 2.1 Object detection sn ojet detetionD the imge is usully snned for ojets of ertin lss @suh s otsA nd ll positions in the imge where mthes re found re mrked y ounding oxes zeliskyD PHIIF sn this thesisD we ddress oth the prolem of oE jet detetion in stti imgesD s well s in dynmi imge sequenesF sn dynmi Chapter 2. State of the art sequenesD the prolem of ojet detetion refers to ojets tht re generlly movE ingD whih enles the use of spei( tehniques for identifying regions of interest tht re not ville in the stti seF husD we divide the ojet detetion lE gorithms ordingly in two prtsF he (rst prt of this setion disusses ojet detetion methods for stti imgesD wheres the seond prt presents ojet deteE tion methods for dynmi sequenesF 2.1.1 Object detection in static images o identify ojets in remotely sensed imgery one n usully rely on mix of spetrl nd geometril informtionF petrl informtion inludes olor ues or intensity edgesD while geometril informtion refers to the shpe of the ojetF heE tetions re otined using olor sed segmenttion lgorithms nd lustering of lol fetures suh s orners nd edgesF st is importnt to note tht ojet deE tetion in high resolution remote sensing imgery hs di'erent hrteristis from tht in ground imgeryX ojets re generlly smll reltive to the ground smpling distneY the kground is lutteredY rottion invrine is requiredF e ommon method for ojet detetion is the sliding window pproh pgeorE giou nd oggioD PHHHD ei nd oD PHIH whih hs een suessfully pplied to humn detetion hll nd riggsD PHHS nd fe detetion iol nd tonesD PHHRF e sliding window is shifted ross the entire imgeF et eh positionD series of feE tures re omputed nd lssi(er is used to return deision vlue whih represents the degree of ertinty tht the imge re inside the window ontins n ojetF sn order to detet ojets of di'erent sles either the size of the window is vriedD or the imge is resled etween the minimum nd the mximum expeted size of the ojetF he nive pproh is to vry the size of the window with one seprtely trined lssi(er model for eh size nd no imge reslingF his pproh is very time onsuming nd requires mny trining smplesF he lterntive pproh is to resle the imge t n di'erent sles nd use (xed window size with single lssi(er model hll nd riggsD PHHSD hollr et lFD PHHWF efter otining ll deiE sion vluesD ojets re deteted y pplying xonEwximum uppression @xwA to these vlues nd using minimum lssi(er unertinty thresholdF fy modeling the ojet in its entiretyD holisti ojet representtion is used in the sliding window pprohF e holisti representtion is usully desired when smll ojets hve to e detetedF he sliding window pproh hs een extensively used in e eril imges for vehile detetionF xguyen et lF PHHU use sliding windows with rr fetures nd orienttion histogrms omined with lol inry ptterns s vehile desriptors nd edfoost preund nd hpireD IWWU for lssi(tionF qszzk et lF PHII only omine sliding windows with rr fetures nd sded edfoost for vehile detetionD whih is very similr to the iol nd tones PHHI pprohF es vehiles ould hve di'erent orienttionsD four disrete orienttions re spei(ed with one lssi(er trined for eh orienttionF urmer et lF PHIQ pply sliding window with ristogrm of yriented qrdient @roqA fetures nd upport etor whines pigure PFIX @A mple imges of irplnes in eril nd stellite imges nd the edge detetion results otined using gnnyEherihe edge detetorF smge y rveen et lF PHISY @A rolisti templte of n irplneF e sliding window pproh n e used to detet irplnes in eril imgesF pigure PFPX rtEsed ojet detetion proedureF pirstD sliding window serh is used to extrt pth imgesF xextD prts re deteted vi sprse representtion of the pthesF rough voting is performed using o'sets of deteted prts of the trget ojet nd (nllyD the mxim re found in the rough imgeF hese mxim represent the possile trgetsF smge y okoy nd swski PHIS @wA to (nd vehilesF pinllyD roi PHIH uses sliding window pproh to detet ots in the open se in stellite imgeryF pigure PFI @A shows n exmple feture @eFgF intensity edgesA tht n e used to detet irplnesF pigure PFI @A shows holisti irplne templte whih hs een onstruted from the geometri fetures of the irplneF hen ojets re su0iently lrgeD prtEsed models n lso e employedF he ide ehind prtEsed models is tht eh prt provides only lol visul properties of the ojet while the geometril rrngement of these prts is hrterized y ertin onnetions etween pirs of prtsF syhologil nd physiologil evidene for prtEsed representtions in the rin vogothetis nd heinergD IWWT s well s ertin omputtionl theories of ojet detetion fiedermnD IWVUD llmnD IWWT support this representtionF en importnt dvntge of prtEsed models is their roustness to prtil olusionsF egrwl nd oth PHHP proposed n pproh for lerning sprseD prtEsed Chapter 2. State of the art representtion of the ojet nd showed its roustness to prtil olusions nd kground vritionF veie et lF PHHV proposed to lern lssEspei( impliit shpe model @swA whih detets the lol ppernes of lss ojets in ordne to ditionry nd then uses rough trnsform to lolize the ojets y onsidering their sptil onsistenyF hng et lF PHIR propose rottion invrint feture y extending roq to enode the fetures of rotted prts nd then luster the prts sed on their distne from eh otherF okoy nd swski PHIS detet oE jet prts y mens of lssEspei( sprse imge representtion of pthes using lerned ojet nd kground ditionries nd rough voting to sptilly integrte the oEourrene of the prtsD whih enles ojet detetionF pigure PFP shows the detetion workE)ow used y okoy nd swski PHISF wrked point proess models hve reeived n inresed ttention in the lst dede for their ility to ope with lrge numer of ojets in lrge dt setsF husD suh models hve een pplied to wide vriety of ojet detetion prolemsF errin et lF PHHS model treeErowns s reliztion of mrked point proess of ellipsesD where the point proess represents the lotions of the trees nd the mrk their geE ometri fetures nd determines the numer of stemsD their position nd their sizeF voste et lF PHHS tkle the prolem of unsupervised line network extrtion suh s rods or rivers in stellite imgesF hey design mrked point proess of line segments nd introdue prior model tht exploits the topologil properties of the networks onsideredF yrtner PHHR detets uildings in dense urn res using mrked point proess of retnglesF hesmps et lF PHHV use mrked point proess model of ellipses to detet )mingos in high resolution stellite imgesF he numer of irds in the popultion is ountedF golor informtion is used for ird detetionF pinllyD (rst model for ot extrtion hs een proposed y fen rdj et lF PHIHF hey spei(lly ddress the prolem of ot detetion in hrorsD whih is very di0ult due to the distriution of the ojetsF hese models re of prtiulr interest throughout this thesis nd the underlying theoretil frmework will e disussed in the next hpterF e relile pperneEsed detetion of ojets remins nevertheless hllenging tsk in eril nd stellite imges rokj nd wedioniD PHIRD espeilly when the ojets re very smllF es opposed to stti imgeryD dynmi imges @or videosA ofE fer dditionl temporl informtion tht n e exploited for ojet detetion tsks in the se of moving ojetsF 2.1.2 Object detection in dynamic images elthough sttionry ojets still hve to e deteted sed on their pperne @this se ws desried in the previous setionAD moving ojets n e deteted sed on their motionF roweverD efore moving ojets n e deteted nd trkedD the mer motion hs to e ompensted forD sine not only the ojetsD ut the entire sene is moving in eril videosF xeverthelessD mer motion ompenstion exeeds the sope of this thesis nd thusD will not e disussedF he interested reder n refer to eutshD PHIS for glol overview of mer motion ompenstion methodsF yne the mer motion hs een ompensted forD independent motion n e deteted @tht isD motion tht is not dependent on the mer motionAF smge di'ereningD kground lerning or lustering of moving lol fetures re exmples of independent motion detetion methodsF hi'erene imges re the most ommon pproh io et lFD PHIHD go et lFD PHIID gherghi nd heikhD PHIPD leemi nd hhD PHIQ nd onsists in omputing the intensity vlue di'erene D t pixel (x, y) in the overlpping re

A o of two registered imges I 1 nd I 2 X D(x, y) = |I 1 (x, y) -I 2 (x, y)|, if (x, y) ∈ A o 0 otherwise @PFIA
trong lol pperne hnges used either y moving ojets or impreise imE ge registrtion re hrterized y high di'erene vluesF he numer of frmes needed to ensure relile result depends on the veloity of the moving ojetD s well s on the sensor frme rteF por exmpleD two onseutive imges re enough for low frmeErte sensors @eFgF 1 -2rzAD s the moving ojet produes prominent motion los in the di'erene imge leemi nd hhD PHIQF sn high frmeE rte sensors @eFgF ≥ 25rzAD prominent los n e otined y onsidering only every nEth imge for omputing the di'erene imge hstry nd hozengerdtD PHHSF roweverD this pproh n led to n e'et known in literture s ghost-ingF qhosting mens tht eh moving ojet produes two motion los in the di'erene imgeD one orresponding to its position in frme I 1 nd distint one orresponding to its position in frme I 2 F his e'et n e hndled y omputing the di'erene imge etween three onseutive frmes io et lFD PHIHD uek et lFD PHIQX

D(x, y) = min(|I 1 (x, y) -I 2 (x, y)|, |I 2 (x, y) -I 3 (x, y)|), if (x, y) ∈ A o 0,
otherwise @PFPA fkground lerning nd foreground segmenttion hs een originlly designed for stti mers irdiD PHHRD fouwmnsD PHIIF fkground lerning nd suE trtion is e'etive when high mount of imges with lrge overlpping re nd smll numer of moving ojets re villeF his is generlly not the se in eril imgesD sine senes my ontin hundreds of moving ojets nd the mer is lso moving suh tht only limited numer of frmes with lrge overlpping res n e extrtedF reneD wellEknown kground modeling methods suh s the tu'erEqrimson tu'er nd qrimsonD IWWW stohsti kground modeling re di0ult to pplyF xeverthelessD erer et lF PHHT uses 30 suessive frmes for tu'erEqrimson kground modeling in the ed qreen flue @qfA olor speD while tones et lF PHHS use similr pproh in the rue turtion lue @rA olor speF wotion los re identi(ed y registering the urrent frme I nd the lerned kE ground BG nd then perform pixelEwise sutrtionD s in the se of di'erene imgesX ) n e mde s followsX

D(x, y) = |I(x, y) -BG(x, y)|, if (x, y) ∈ A o 0,
p t|t-1 (X|Z 1:t-1 ) = f t|t-1 (X|X t-1 )p t-1|t-1 (X t-1|Z 1:t-1 )µ s (dX t-1 ), @PFTA
where f t|t-1 (X|X t-1 ) is the wrkov stte trnsition funtion nd µ s (•) is the domiE nting mesure of the stte speF yne the mesurements Z t t time t re otinedD the posterior n e updted s followsX

p t|t (X|Z 1:t ) = g t (Z t |X)p t|t-1 (X|Z 1:t-1 ) g t (Z t |X)p t|t-1 (X|Z 1:t-1 )µ s (dX t ) , @PFUA
where g t (Z t |X) is the multiple ojet likelihood funtionF st n e oserved tht the only di'erene to the singleEtrget fyes (lter lies in the ft tht X t nd Z t re p with vrile dimension tht depends on tF he omplexity of the multiple ojet fyes (lter grows exponentilly with the numer of ojetsF husD there is need for pproximte solutionsF he roility rypothesis hensity @rhA (lter is the hepest trtle pproxiE mtion of the optiml multipleEojet fyes (lter proposed y whler PHHQ tht is sed on the p frmeworkF he rh (lter is sed on the following ssumptionsX

• rgets evolve nd generte mesurements independently from eh otherY

• he irth p is independent from the urrent or surviving trgets nd is onsidered to e oissonY

• xoise is oisson pD independent from the mesurements pY

• he predited nd posterior multiple ojet p re pproximted s oisson psF sf we denote the intensity funtions ssoited to the predited nd the posterior multiple ojet stteD γ t|t-1 (x) nd γ t|t (x) respetivelyD we n write the predited stte s followsX sn the previous hpterD rod overview of urrent tehniques to detet oE jets in stti nd dynmi imge sequenes nd to trk them in time hs een presentedF xeverthelessD in the lst dedeD mrked point proesses hve reeived gret ttention for ojet detetion in lrge dt setsD with emphsis in iologil imgery nd remotely sensed stellite dtF his hpter provides short desription of the theoretil frmework of point proE esses in generl nd the neessry ingredients to onstrut fully utomted ojet detetion model re disussedF he hpter is divided into three prtsX the point proess fundmentls re presented in the eginning of the hpterF es suh models generlly depend on set of hyperEprmeters whih re di0ult to e set mnullyD prmeters estimtion tehniques re presented nextF pinllyD trditionl optimizE tion proedures nd reent developments nd improvements thereofD re disussed t the end of this hpterF 3.1 Fundamentals sf we onsider the imge support to e KD with K ⊂ R d D then on(gurtion of points x is (nite unordered set of points in KF he on(gurtion speD ΩD n then e written sX Ω = n∈N Ω n , @QFIA pigure QFIX imultion of oisson point proessF @A romogeneous oisson point proessY @A snhomogeneous oisson point proessF

γ t|t-1 (x) = p s,t (u)f t|t-1 (x|u) + b t|t-1 (x|u) γ t-1|t-1 (u)du + µ t (
where

Ω 0 = ∅ nd Ω n = {{x 1 , • • • x n }, x i ∈ K,
∀i} re the set of on(gurtions with no points nd n unordered pointsD respetivelyF he veesgue mesure on KD denoted Λ(K)D is used to mesure the on(gurtion spe s followsX

ν(Ω) = ∞ n=0 ν(Ω n ) = ∞ n=0 Λ(K) n n! = exp Λ(K) .
@QFPA e on(gurtion is lled lolly (nite if it ples t most (nite numer of points in ny ounded forel set A ⊆ KF he fmily of ll lolly (nite on(gurtions is denoted y N lf = N lf K F e point proess n e de(ned sX Denition 3.1.1 (Point process) A point process on K is a mapping X from the probability space (Ω, A, P ) into N lf such that for all bounded Borel sets A ⊆ K, the number N (A) = N X (A) of points falling in A is a (nite) random variable. sn other wordsD point proess refers to ny rndom vrile whose reliztions re rndom on(gurtions of pointsF he most wellEknown point proess is the oisson point proessF Denition 3.1.2 (Poisson point process) A point process X dened on K, with intensity measure µ and intensity function ν, is a Poisson point process if:

• N (A) is Poisson distributed with mean µ(A);

• conditional on N (A), the points in X A are i.i.d. with the density proportional to ν(u), u ∈ A.

he oisson proess is lled homogeneous if ν(u) is onstnt for ll u ∈ KF he oisson proess is model of omplete sptil rndomness or no intertionD sine X A nd X B re independent whenever A, B ∈ K re disjointF pigure QFI shows the reliztion of homogeneous @A nd nonEhomogeneous @A oisson point proessF por every forel set AD the proility mesure π ν (A) ssoited with oisson proess is given yX

π ν (A) = exp -ν(K) 1(∅ ∈ A) + ∞ n=1 π νn (A) n! @QFQA with π νn (A) = K • • • K 1({x 1 , • • • , x n } ∈ A n )ν(dx 1 ) • • • ν(dx n ) @QFRA
where A n is the suset of on(gurtions in A tht ontins extly n points nd

1 [•] is the inditor funtion @1(true) = 1; 1(f alse) = 0AF
he intensity funtion ν(•) of the oisson proess n e used to ontrol the point density of the on(gurtionsF xeverthelessD no orreltion etween pointsD nor onstrints on their reltive positionsD n e enfored using this simplisti modelF o model suh onstrintsD point proess n e de(ned y density funtion wFrFtF the oisson mesureF Denition 3.1.3 (Density of a point process) The probability density, f , of a point process w.r.t. the π ν (•) law of a Poisson process is a mapping from the conguration space Ω into [0, ∞[ such that:

f : Ω → [0, ∞[, Ω f (x)dπ ν (x) = 1.
@QFSA por every forel set A in ΩD the proility mesure on Ω tht de(nes point proess is de(ned y P (A) = A f (x)dπ ν (x)F en equivlene n e drwn etween the proility densityD f D of point proess nd the elief density p Ξ of rndom (nite setD ΞD y interpreting the ltter s the donExikodyn derivtive of the orresponding distriution P Ξ wFrFtF the domintE ing mesure π ν F fy introduing the density funtion wFrFtF the mesure of oisson proessD interE tions etween points n now e e0iently modeledF roweverD longErnge orE reltions re hevy to implementF he wrkov property is very useful in hndling this implementtion detil s it restrits the longErnge orreltions to lol neighE orhoodF sn order to de(ne the wrkov property for point proessesD the notion of neighorhood is neededF Denition 3.1.4 (Neighborhood) The neighborhood ∂(A) of a set A ⊆ K is dened as

∂(A) = {x ∈ K : x ∼ a for some a ∈ A} @QFTA
where ∼ is a reexive and symmetric relation on K.

Chapter 3. Using Point processes for object detection fsed on the neighorhood reltionD the wrkov proess n e de(ned s followsX Denition 3.1.5 (Markov process) Let X be a point process with density f . X is a Markov process under the symmetric and reexive relation ∼ if and only if, for every conguration x ∈ Ω such that f (x) > 0, X satises

• f (y) > 0 for every y ⊂ x;

• for every point u from K, f (x∪u)/f (x) only depends on u and its neighborhood

∂({u}) ∩ x = {x ∈ x : u ∼ x}.
he rmmersleyEgli'ord theorem llows the density of the wrkov proess to e deomposed s the produt of lol funtions de(ned on liquesX The Papangelou conditional intensity for a point process X with density f is dened by

λ * (x, u) = f (x ∪ {u}) f (x) , x ∈ N lf , u ∈ K \ x,
where we consider a/0 = 0, for a ≥ 0.

sndeedD the pngelou onditionl intensity does not depend on the normlE izing onstnt whih nels out nd thusD λ * (x, u) does not depend on itF por oisson proessD the pngelou onditionl intensity does not depend on x either @λ * (x, u) = f (u)D sine the points re sptilly independentAF he onditionl inE tensity n e used to distinguish etween di'erent types of point proessesF reneD pigure QFPX imultion of truss intertion proess in squre window @l = 10A with (xed β = 10 nd rdius r = 1F @A γ = 0.0Y @A γ = 0.5Y @A γ = 1.0F ttrtive point proesses n e used to model lustering phenomenD suh s the nturl lustering of plnts in n reF sn ontrstD repulsive point proesses n e used to model for exmple the repulsive ehvior of )ying irplnesF por sfety reE sonsD irplnes re required to )y t ertin distne from one notherD exhiiting thus repulsive ehviorF ettrtive or repulsive point proesses n e desried sed on the onditionl intensity s followsX Denition 3.1.7 (Attractive vs Repulsive point process) A point process X is called attractive if λ * (x, u) ≤ λ * (y, u), whenever x ⊂ y or repulsive if λ * (x, u) ≥ λ * (y, u), whenever x ⊂ y.

e typil exmple of wrkov point proesses is the pirwise intertion proessF sn this seD the unnormlized density funtion is written sX

h(x) = n(x) i=1 b(x i ) 1≤i<j≤n(x):x i ∼x j g(x i , x j ), @QFWA
with n(x) eing the numer of points in the on(gurtion xD b(x i ) is unry funtion of the point x i nd g(x i , x j ) is n intertion funtionD eFgF nonEnegtive funtion for whih the right hnd side of eqF QFW is integrle with respet to the unit rte oisson proessF

Example 3.1.1 (The Strauss process) A simple pairwise interaction process is the Strauss process given by the density:

f (x) = β n(x) γ s(x) , @QFIHA
where β > 0 and s(x) represents the number of pairs of points in x that are at a distance of r or less apart. • f (y) > 0 for all y ⊆ x;

• for all (u, l) ∈ W, f (x ∪ {(u, l)})/f (x) depends only on (u, l) and ∂({(u, l)}) ∩ x = {(v, k) ∈ x : (u, l) ∼ (v, k)}.
por imge nlysis purposesD the mrk is used to de(ne the geometry of the ojetD while the points in the on(gurtion refer to the enter of the ojets hesomes et lF PHIIF sn terms of time sequene nlysisD the mrks n inlude not only geometri feturesD ut lso leling informtionF he truss proess exmple shows n importnt property of point proessesF fsed on the vlues of the prmeters of the proessD it n produe signi(ntly di'erE ent outputsF reneD suitle prmeter setting is required for given prolemF elthough these prmeters ould e set mnullyD utomti prmeter estimtion tehniques re more desirle s they n e pplied to lrge vriety of dt sets ontining huge mounts of dt without the need of humn interventionF reneD suh tehniques will e disussed in the following setionF

, i = 1, • • • , n} that exists in K × M is formed from a collection of circles with dierent colors, x i = (k i , m i ), with k i ∈ K and m i ∈ M .
The interaction process is dened by:

f (x) = α (k,m)∈x β(k, m) (u,i)∼(v,j)∈x γ ij (||u -v||). @QFIIA
The homogeneous Poisson point process with unit intensity is the reference measurement for the point positions, to which the marks distributed according to ν M are independently associated. The normalizing constant is denoted here with α and

(u, i) ∼ (v, j) ↔ ||u -v|| ≤ r, u = v. @QFIPA
The model is well dened if β : K × M → R + and γ ij : (0, r] → [0, 1] for all i, j ∈ M and r > 0.

The three sets of parameters for this model are as follows:

1. Parameters relative to the probability density of the mark are given by M and ν M . These parameters are used to determine the distribution of the mark, as well as the mark itself;

2. Parameters relative to the interaction elds of the objects are given by the radius r which determines the maximum distance between objects at which they interact with each other;

3. Parameters relative to the interactions between the objects are given by the activity parameter β and the symmetric interaction function γ ij = γ ji , ∀i, j ∈ M .

sn order to implement fully utomti systemD ll of these prmeters should idelly e estimted utomtillyF his setion introdues some of the most ommon pprohes to prmeter estimtion in the ontext of mrked point proessesF e more detiled overview on prmeter estimtion in the ontext of stohsti proesses is given y peldmn nd girio PHHWF ine the ojet on(gurtion x is unknown nd the imge likelihood f θ (y)D with θ eing the prmeter vetorD hs no expliit expressionD the extended likelihoodD f θ (x, y)D iFeF the joint likelihood of the oserved dtD yD nd the hidden dt or the on(gurtion of ojets tht we wnt to retrieveD xD n e usedF 3.2.1 Maximum likelihood estimation using MCMC techniques sn this estimtion method @short wvwgwg elillo nd rnokD IWWUAD insted of onsidering diretly the logElikelihoodD the rtio etween the likelihood for the urrent prmeterD θD nd the likelihood for (xed prmeterD ψ ∈ Θ is onsideredF por the generl se of nonEnormlized density funtion h θ (z)D this rtio n e expressed in terms of the expettion of the nonEnormlized densitiesX

c(θ) c(ψ) = 1 c(ψ) h θ (z)µ(dz) = 1 c(ψ) h θ (z) h ψ (z) h ψ (z)µ(dz) @QFIQA = h θ (z) h ψ (z) f ψ (z)µ(dz) = E ψ h θ (Z) h ψ (Z)
. @QFIRA he normlizing onstnt for the oservtion likelihood f θ (y) n e written sX c(θ) = h θ (x, y)µ(dx)µ(dy). @QFISA imilrlyD the normlizing onstnt ssoited to the onditionl density f θ (x|y) of the hidden dt xD given the dt yD n e written sX c(θ|y) = h θ (x, y)µ(dx). @QFITA husD the oservtion likelihood n e written sX

f θ (y) = 1 c(θ) h θ (x, y)µ(dx) = c(θ|y) c(θ) @QFIUA
nd the rtio of the logElikelihoods n e expressed sX

l(θ) = log c(θ|y) c(θ) -log c(θ) c(ψ) @QFIVA
sing eqF QFIRD we otin the following expression for the rtio of the logElikelihoodsX Repeat:

l(θ) = log E ψ h θ (X, Y ) h ψ (X, Y ) |Y = y -log E ψ h θ (X, Y ) h ψ (X, Y ) . @
IF @iAEstepX ompute Q(θ, θ i ; y) = E θ i [log f θ (X, Y )|Y = y], @QFPHA
where θ i is the vlue t itertion i of the prmeter vetor θ tht needs to e estimtedY PF @wAEstepX θ i+1 = arg max θ Q(θ, θ i ; y) IF given on(gurtion xD the pseudoElikelihood is funtion very lose to the likelihood funtion @nd in some ses the two re the smeD suh s in the trivil se of oisson point proessAY PF the estimtors for the mximum likelihood nd the mximum pseudoE likelihood hve similr vlues for the point proesses of interestF he pseudoElikelihood of point proess xD given the oservtion y n e de(ned s wteu nd wontesD PHHIX

i = i + 1 until |θ (i+1) -θ (i) | ≤ ε 3.2.2
P L(θ; x, y) = x i ∈x λ θ (x i ; x, y) exp -λ θ (u; x, y)Λ(du) , @QFPIA
where Λ(•) is the intensity mesure of the referene oisson point proess nd λ θ (•) is the pngelou onditionl intensity funtion de(ned in eqF QFIFT for the prmeter vetor θF he pseudoElikelihood n e used to pproximte the likelihood of on(gurtion y onsidering the ojets within tht on(gurtion to e independent from eh otherF he pseudoElikelihood n e inserted in the expettion step of the iw lgorithm to filitte omputtionsF sn the followingD stohsti versions of the iw re disussedF Algorithm 2 tohsti ixpettion wximiztion lgorithm proposed y geleux nd hieolt IWVSF Inputs E initil vlue θ 0 D imge y i = 0

Repeat:

IF @AEstepX imulte x i ∼ f θ i (x, y)Y PF @iAEstepX gompute Q(θ, θ i ; y) = log P L(θ; x i , y)Y QF @wAEstepX θ i+1 = arg max θ Q(θ, θ i ; y) i = i + 1 until |θ (i+1) -θ (i) | ≤ ε 3.2.2.2 Stochastic EM
he most nturl pproh for stohsti version of the iw lgorithm is to use wonte grlo methods to pproximte the likelihood in eqF QFPHF his ide ws proE posed y ei nd nner IWWH nd is known in literture under the nme Monte-Carlo Expectation Maximization @wgiwAF he expettion step is thus omputed s followsX IF simulte n on(gurtions x (1) , • • • , x (n) D onditioned y the oservtion y unE der the prmeter vetor θ i nd PF ompute for eh on(gurtion x (j) the logrithm of the joint likelihood f θ i (x (j) , y) nd pproximte the likelihood with the empiril men of the n on(gurtionsF he resemlne to the wvwgwg lgorithm n e oservedF xeverthelessD wgiw hs signi(nt dvntge in tht there is no need to simulte the oservtions yF yn the other hndD this method hs signi(nt drwk onsisting in the omE puttion ost needed to simulte n on(gurtions nd pproximting the likelihood for eh one of themF reneD nother stohsti version of the iw lgorithm ws proposed y geleux nd hieolt IWVSF heir ide ws to reple the omputtion of the likelihood in eqF QFPH with muh esier omputtion of the logElikelihood

Q(θ, θ i ; y) = log f θ i (x i , y)D
where the on(gurtion x i is simulted onditioned on the oservtions y under the prmeter vetor θ i F his pproh is known in literture under the nme tohsE ti iw @iwA nd is prtiulr se of the wgiw lgorithmD where the numer of simulted on(gurtions is (xed to n = 1F fesides the deresed omputtionl ostD the iw presents signi(nt theoretil dvntge ompred to the iwF qiven its stohsti ntureD iw is less likely to onverge to lol mximumD insted of glol oneF he iw is presented in Repeat:

IF @AEstepX imulte x i ∼ f θ i (x, y)Y PF @iAEstepX gompute Q(θ, θ i ; y) = (1 + τ i )Q(θ, θ (i-1) ; y) + τ i n n j=1 log P L(θ; x j i , y)Y QF @wAEstepX θ i+1 = arg max θ Q(θ, θ i ; y) i = i + 1 until |θ (i+1) -θ (i) | ≤ ε
elgorithm PD where the likelihood in eqF QFPH is repled y the pseudoElikelihood desried in eqF QFPIF et nother vrint is the tohsti epproximtion iw @eiwA lgorithm proE posed y heylon et lF IWWW s n lterntive to the stohsti versions previously mentionedF he min di'erene in the eiw is the wy in whih the expettion is omputedD whih is sed on the stohsti pproximtion method proposed y oins nd wonro IWSIF he expettion thus orresponds toX

Q(θ, θ i ; y) = (1 + τ i )Q(θ, θ (i-1) ; y) + τ i n n j=1 log P L(θ; x j i , y), @QFPPA
where τ i is the forgetting ftor nd n is the numer of on(gurtions simultedF his pproh keeps memory of pst simultionsF he forgetting ftor hs ruil in)uene on the onvergene properties of the eiwF e good vlue for τ i is τ i = j -α D with α ∈ (0.5, 1) @tnkD PHHTAF he eiw is presented in lgorithm QD with n = 1F he eiw hs een reently proposed y foisunon nd erui PHIR for pE rmeter estimtion for the spei( prolem of ot extrtion in hrorsF yne the prmeters of the point proess model hve een estimtedGlernedD optimiztion methods n e pplied to (nd the est on(gurtion of ojets for given imgeF sn the following setionD pproprite optimiztion methods for the point proess frmework re disussedF

Optimization

his setion tkles the prolem of mrked point proess simultionF es opposed to point proessD the intensity mesure of mrked point proess is written s produt of two mesuresD one mesuring forel sets on the imge support K nd one mesuring forel sets on the mrk spe M F reneD the intensity mesure of mrked oisson point proess n e written sX

ν(•) = (Λ K × P M )(•) @QFPQA
where Λ K is the veesgue mesure on K nd P M is some proility mesure on M F he interest lies in (nding the most prole on(gurtion of ojetsD xD tht (ts the given dt yF hereforeD n optimiztion prolem hs to e solvedF purtherE moreD point proess models represent prtiulr type of optimiztion prolems in whih the numer of vriles to e estimted is itself rndom vrileF husD dimensionl jumps re neessry to reh n optiml solutionD mening tht the optimiztion proedure must e le to dd or remove ojets from on(gurtionF he lssil wrkov hin wonte grlo smplers @oert nd gsellD PHHSA deE veloped to tkle trnsEdimensionl simultion prolemsD s well s reent developE ments in this re nd lterntive lgorithms re introdued in this setionF wrkov hin wonte grlo @wgwgA smpling ws invented in the SHs shortly fter the ordinry wonte grlo lgorithm in vos elmosD one of the few ples where omE puters existed t the timeF sn their ttempt to simulte )uid dynmisD wetropoE lis et lF IWSQD relized tht they do not need to simulte the ext dynmisD sine it su0ed to simulte wrkov hin with the sme equilirium distriutionF he lgorithm eme known s the Metropolis algorithm nd gined populrity s omputers eme widely villeF vterD rstings IWUH generlized the lE gorithm nd trnsformed it into wht eme known s the Metropolis-Hastings algorithmF pinllyD qreen IWWS generlized further the wetropolisErstings lgoE rithm y introduing dimensionl jumps into the lgorithm whih eme known s the Metropolis-Hastings-Green algorithm or lterntively Reversible Jump MCMC algorithm @twgwgAF he min ide of this lgorithm is to simulte wrkov hin (X i ) i∈N over the on(gurtion spe Ω whih onverges to trget distriution πF xeverthelessD the wrkov hin must possess ertin properties in order to gurntee its onvergene to the desired trget distriution πF

The reversible jump MCMC sampler

fefore desriing the smpler generlized y qreen IWWSD the neessry theoretil foundtion is presentedF 3.3.1.1 Markov chains and convergence properties Denition 3.3.1 (Markov chain) Let (X n ) be a sequence of random variables, with values taken from a space Ω associated with its sigma-algebra B. (X n ) is said to be a Markov chain if:

p(X i+1 ∈ A|X i = x i , • • • , X 0 = x 0 ) = p(X i+1 ∈ A|X i = x i ), ∀A ∈ B. @QFPRA
eording to the de(nitionD the evolution of wrkov hin depends only on the urrent stteF Denition 3.3.2 (Homogeneous chain) If the evolution of a Markov chain does not depend on the time parameter i, then the chain is said to be homogeneous.

ynly homogeneous wrkov hins will e onsidered from now onF sn order for the wrkov hin to evolveD perturtion @or trnsitionA kernel hs to e ssoited to itF Denition 3.3.3 (Perturbation kernel) A perturbation kernel is a function P dened on Ω × B such that:

• ∀x ∈ Ω, P (x, •) is a probability measure; • ∀A ∈ B, P (•, A) is measurable.
A transition kernel P associated to a Markov chain (X i ) i∈N is given by:

P (x, A) = p(X i+1 ∈ A|X i = x). @QFPSA
eprt from the homogeneity onstrintD in order to suessfully simulte wrkov hin using n wgwg smplerD the hin must eX

• StationaryF e mesure π is sttionry for the wrkov hin of perturtion kernel P ifX

π(A) = P (x, A)dπ(x), ∀A ∈ B. @QFPTA
sndeedD sttionry mesure π is needed for the hin to onvergeF

• ReversibleF e wrkov hin is reversile if its perturtion kernel P stis(es the equlityX

B P (x, A)dπ(x) = A P (x , B)dπ(x ), ∀A, B ∈ B. @QFPUA
imply putD the proility of going from B to A under π is the sme s going from A to BF

• IrreducibleF e wrkov hin is irreduile if for ll pirs of sttes (x, x )D

there is positive proility of rehing x from x in (nite numer of trnsitionsX

∀x, x ∈ Ω, ∃m < ∞ : p(X i+m = x|X i = x ) > 0.
@QFPVA srreduiility mens tht the hin hs stritly positive proility of rehE ing ny πEprole set in (nite timeD independent of the initil onditionsF

• AperiodicF he period d of wrkov hin is de(ned sX

d = gcd{k : p(X k = x|X 0 = x) > 0}. @QFPWA
where gcd{•} is the gretest ommon divisorF e wrkov hin is periodi if d = 1F e wrkov hin needs to e periodi in order to onvergeF

• Harris recurrentF e wrkov hin is rrris reurrent if

p(∃t : X t ∈ A|X 0 = x) = 1 ∀x ∈ Ω, ∀A ∈ B : π(A) > 0. @QFQHA
he rrris reurrene property is needed to ensure tht the wrkov hin onvergesD regrdless of the initil onditionsF

• ErgodicF e wrkov hin with sttionry mesure π onverges ergodi toE wrds π if it is periodi nd rrris reurrentF he wrkov hin needs to e designed so s to e ergodiD in order to onverge to the desired distriution πF sn his workD qreen IWWS proposes the use of mixture of kernels to simulte wrkov hinF

3.3.1.2 State dependent mixing gonsider (nite set of perturtion kernels Q m (x, A)D m ∈ M D suh thtX Q(x, A) = m∈M Q m (x, A). @QFQIA
he perturtion kernels must stisfy the following onstrintsX

• Q m (x, Ω) is knownD ∀m ∈ M Y • m∈M Q m (x, Ω) ≤ 1D ∀x ∈ ΩY • there exists symmetri mesure φ m (dx, dx )D ∀m ∈ M D φ m (•, •) : Ω × Ω suh tht π(dx)Q m (x, dx ) is solutely ontinuous wFrFtF φ m (dx, dx )Y
• ∀m ∈ M D ∀x ∈ ΩD reliztions n e simulted strting from the normlized perturtion distriutionX

P m (x, •) = Q m (x, •) Q m (x, Ω) . @QFQPA • he ssoited donExikodym derivtive is denoted y f m (•, •) nd is written sX f m (x, x ) = π(dx)Q m (x, dx ) φ m (dx, dx ) . @QFQQA
por given stte X i = xD we n write the updting sheme s followsX 48 Chapter 3. Using Point processes for object detection

IF ith proility Q m (x, Ω) hoose kernel Q m or with proility 1 - m Q m (x, Ω) let the stte unhnged X i+1 = xY PF imulte x ording to the normlized kernel hosenX x ∼ Q m (x, •) Q m (x, Ω) @QFQRA QF gompute the qreen rtioX R m (x, x ) = f m (x , x) f m (x, x ) @QFQSA RF eept the perturtion with proility α m (x, x ) = min(1, R m (x, x )) or rejet otherwiseF 3.3.1.3 Detailed balance
por the hin to e reversile we require tht the proility of the hin in generl set A nd moving to generl set B to e the sme with A nd B reversed @see eqF QFPUAF his is lso known s the detailed balance condition @hfAF sn order to prove the hf ondition for kernel P (•, •)D it su0es to show tht eh suEkernel P m (•, •) mintins the hf ondition qreenD IWWSD yrtnerD PHHRD whereX

P (x, A) = A Q m (x, dx )α m (x, x ). @QFQTA
sing eqF QFQQD we n writeX

A B π(dx)Q m (x, dx )α m (x, x ) = A B f m (x, x )α m (x, x )φ m (dx, dx ). @QFQUA fy de(nition of α m (x, x ) we hve thtX f m (x, x )α m (x, x ) = f m (x , x)α m (x , x). @QFQVA he symmetry of φ m (•,
•) together with this property give the hf ondition in eqF RFTF

Standard perturbation kernels

he hoie of perturtion kernels is n importnt point when using wgwg smE plersF sn this setion we present the most ommon perturtion kernels used in literture qreenD IWWSD yrtnerD PHHRD hesomes et lF PHII nd show tht the detiled lne @hfA ondition is mintinedF he hf for these kernels hs lredy een proven y yrtner PHHRF e reiterte the proof for the irth nd deth kernel nd only stte the results otined y yrtner PHHR for the other kernelsF tndrd perturtion kernels for simulting mrked point proess models n e tegorized s followsX

• Birth and DeathX he irth nd deth kernel is used to dd or remove ojets from on(gurtion nd heneD modify the dimension of the on(gE urtion speF his kernel is in itself mixture of two suEkernelsX irth kernel @used to dd new ojet to on(gurtionA nd the deth kernel @used to remove n ojet from on(gurtionAF sf we suppose tht the irth kernel genertes new ojet in W ording to the proility lw ν(•) ν(W ) D where ν(W ) is mesure on W D nd the deth kernel uniformly hooses one ojet to e deleted from the urrent on(gurtionD then we n write the kernel s followsX

Q(x, •) = p b (x)Q b (x, •) + p d (x)Q d (x, •), @QFQWA
with the irth kernel de(ned yX

Q b (x, A) = u∈W 1 A (x ∪ u) ν(du) ν(W ) A ∈ N lf @QFRHA nd Q d (x, A) = u∈x 1 A (x \ u) 1 n(x) , @QFRIA exept if n(x) = 0D when Q d (x, •) = I(x, •)F
Proof of DBF vet us onsider the following mesure φD where A nd B re two mesurle susets of CX

φ(A×B) = C u∈W 1 A (x)1 B (x ∪ u)ν(du)µ(dx)+ C 1 A (x) u∈x 1 B (x \ u)µ(dx).
@QFRPA he symmetry of this mesure omes from the ft tht ν(•) is the intensity mesure of the underlying oisson proess governed y the lw µ(•)F vet A n = A ∩ N lf n where N lf n is the suset of N lf W orresponding to on(gurtions ontining extly n ojetsF henD y de(nition of the oisson point proess we otinX

φ(A n × B n-1 ) = e -ν(W ) n! W n u∈x 1 An (x)1 B n-1 (x \ u)ν n (dx) = e -ν(W ) n! W n n1 An ({x 1 , • • • , x n })1 B n-1 ({x 1 , • • • , x n-1 })dν n (x) = e -ν(W ) (n -1)! W n-1 W 1 B n-1 (x )1 An (x ∪ u)ν n-1 (dx )ν(du) = φ(B n-1 × A n ).
he symmetry of φ is otined y writing

φ(A, B) s n in(nite sum φ(A, B) = (φ(A n , B n-1 ) + φ(A n , B n+1 ))F xote tht A × B hs stritly positive meE sure π(•)Q(•, •) nd
tht its mesure φ is lso stritly positiveF e now hve to show tht φ(dx, dx ) domintes π(dx)Q(x, dx ) nd ompute the donE xikodym derivtiveF here re two ses to onsiderX BirthX sf x = x ∪ u we otin the following expressionsX

π(dx)Q(x, dx ) = h(x)µ(dx)p b (x) ν(du) ν(W ) φ(dx, dx ) = µ(dx)ν(du) @QFRQA
where h(x) is the nonEnormlized density funtion of x nd from where we n dedue the domintion of φ(dx, dx

) over π(dx)Q(x, dx )F he donE xikodym derivtive yieldsX f (x, x ) = p b (x) h(x) ν(W ) . @QFRRA DeathX sf x = x \ u we otin the following expressionsX π(dx)Q(x, dx ) = h(x)µ(dx)p d (x) 1 n(x) φ(dx, dx ) = µ(dx) @QFRSA
where h(x) is the nonEnormlized density funtion of x nd from where we n gin dedue the domintion of φ(dx, dx )F he donExikodym derivtive in this se if given yX

f (x, x ) = p d (x) h(x) n(x) @QFRTA
his ends the proofF Green ratioF he qreen rtio hs two di'erent expressionsD depending on whether n ojet is dded or removed from the on(gurtionF sn se of Birth @x = x ∪ uAD the qreen rtio is given yX

R(x, x ) = f (x , x) f (x, x ) = p d (x ) p b (x) h(x ) h(x) ν(W ) n(x ) . @QFRUA sn se of Death @x = x \ uAD the qreen rtio is given yX R(x, x ) = f (x , x) f (x, x ) = p b (x ) p d (x) h(x ) h(x) n(x) ν(W ) .
@QFRVA e vrint of this perturtion kernel is the Birth and Death in a Neigh-borhoodD whih llows the retion or removl of n ojet in the prede(ned neighorhood of n existing ojetF por the type of pplitions onsideredD we n ssume tht if n ojet exists t given lotionD it is more likely tht dditionl ojets exist in tht reF he firth nd heth in xeighorhood kernel inorportes this ssumption y proposing the retion of ojets in res where other ojets lredy exist nd s suhD inreses the onvergene of the wrkov hinF vet ∼ e symmetri reltion on

W suh tht ∀u, v ∈ W u = vX u ∼ v ↔ d K (u, v) ≤ d max , @QFRWA where d K (•, •) is the iuliden distne on KF vet N (x) e the set of pirs of relted ojetsX N (x) = {{u, v}, u, v ∈ x sFtF u ∼ v}. @QFSHA he neighorhood of n ojet u is de(ned sX ∂(u) = {v ∈ W sFtF v ∼ u}, ∂(u) ⊆ W @QFSIA
nd the neighorhood of on(gurtion is de(ned sX

∂(x) = {u ∈ W : u ∼ v for some v ∈ x}. @QFSPA
he irth nd deth kernel in neighorhood n e written sX

Q(x, •) = p b (x)Q b (x, •) + p d (x)Q d (x, •) @QFSQA
he irth in neighorhood kernel is written sX

Q b (x, A) = u∈x η x b (u)Q u b (x, A), @QFSRA where η x b (u) is the disrete distriution ording to whih new ojet is proposed nd Q u b (x,
•) dds ojets to the neighorhood of uF he deth in neighorhood kernel is written sX

Q d (x, A) = u∈x η x d (u)1 A (x \ u), @QFSSA
where η x d (u) is null for ny ojet u not elonging to ny pir in ∂(x)F es the irth nd deth in neighorhood is speil se of the previously introdued irth nd deth kernelD we n keep the sme symmetri mesure φ desried in eqF QFRPF BirthX fefore we stte the donExikodym derivtiveD we rie)y present how to generte neighor of n lredy hosen ojet uF pirstD vetor z is generted ording to the lw of the rndom vrile Z on the spe ΣF eondD n injetion ξ u (•) is pplied on z with

ξ u : Σ → W @QFSTA z → v. @QFSUA he pir @z, ξ u A must give n ojet v tht is neighor of u nd thus ξ u (Σ) = ∂(u)F sf we suppose tht Z follows lw P Z on ΣD we n detil Q u b X Q u b (x, A) = P Z (x ∪ ξ u (Z) ∈ A) = P Z (ξ u (Z) ∈ A x )
where its donExikodym derivtiveD for whih he tkes the uniform lw s n exmpleX f

A x ⊆ W orresponds to the set A x = {v ∈ W sFtF x ∪ v ∈ A} whih llows us to writeX Q u b (x, A) = Σ 1 Ax (ξ u (z))dP Z (z).
Z (•) = 1 λ Z (Σ) F
yrtner PHHR shows tht under these hypothesesD the donExikodym derivE tive of the irth kernel n e written sX

f (x, x ∪ v) = p b f ν (v) h(x) u∈x η x b (u)f Z (ξ -1 u (v))Λ u (v) , @QFSWA with Λ u (v) = 0 if v / ∈ ∂(u) = ξ u (Σ) |J ξ -1 u (v)| otherwise @QFTHA where J ξ -1 u (•) is the toin of ξ -1 u (•)F
DeathX he expression of the donExikodym derivtive otined y yrtner

PHHR isX f (x, x \ u) = h(x)p d η x d (u). @QFTIA
Green ratioF es in the se of the uniform irth nd deth kernelD the qreen rtio hs two di'erent expressionsD depending on whether n ojet is dded or removed from the on(gurtionF sn se of birth @x = x ∪ vAD the qreen rtio hs the formX

R(x, x ∪ v) = h(x ∪ v) h(x) p d (x ∪ v) p b (x) η x∪v d (v)f ν (v) u∈x η x b (u)f Z (ξ -1 u (v))Λ u (v)
@QFTPA sn se of death @x = x \ vAD the qreen rtio hs the formX

R(x, x \ v) = h(x \ v) h(x) p b (x \ v) p d (x) u∈x\v η x\v b (u)f Z (ξ -1 u (v))Λ u (v) η x d (v)f ν (v)
. @QFTQA

• Local perturbationsX hese perturtions do not hnge the dimension of the on(gurtion speD ut rther modify the prmeters of n ojet in the urrent stteF ypil exmples inludeX rottionD trnsltion nd slingF pigure QFQ depits these type of perturtionsF yrtner PHHR disusses these kernels in detilF e will rie)y mention the min results otined y yrtner PHHRF vet Σ e ountle suset of R d nd s(•) the ssoited veesgue mesureF qiven on(gurtion of ojets x nd n ojet uD let Z (x,u) e rndom pigure QFQX ixmple of modi(tion of the prmters of n ellipse with lol perturtion kernelsY @leftA ottionY @middleA rnsltionY @rightA leF vrile living on set Σ(x, u)

⊂ ΣF henote P x,u Z (•) the distriution of Z (x,u) on Σ nd f (x,u) Z (•) the ssoited density funtionF pinllyD onsider n injetion ζ x : W × Σ → W @QFTRA (u, z) → v. @QFTSA henD for urrent on(gurtion x = {u i }, i = 1, n(x)D the lol perturtion kernel n e desried s followsX IF ghoose n ojet u ∈ x ording to disrete proility lw η x (u i )Y PF qenerte z using the distriution of Z (x,u) Y QF gompute v = ζ x (u, Z)Y RF ropose x = x \ u ∪ vF he perturtion kernel n e written sX Q(x, A) = u∈x η x (x, u)P (x,u) Z (1 A (x \ u ∪ ζ x (u, z))).
@QFTTA e n onsider the following mesureX

φ(A × B) = A u∈x Σ(x,u) 1 B (x \ u ∪ ζ x (u, z))ν(dz)µ(dx). @QFTUA
o otin the symmetry of this mesureD we suppose the symmetry of the trnsformtionX

v = ζ x (u, z) ⇔ ∃z ∈ Σ(x , v) sFtF x = x \ u ∪ v, u = ζ x (v, z). @QFTVA he uniqueness of z omes from the injetion ζ x (u, •)F
Green ratioF he following qreen rtio is otinedX

R(x, x ) = f (x , x) f (x, x ) = h(x ) h(x) η x (v) η x (u) f (x ,v) Z (z) f (x,u) Z (z)
. @QFTWA 54 Chapter 3. Using Point processes for object detection 3.3.1.5 Reversible jump MCMC he reversile jump wgwg smpler introdued y qreen IWWSD desries n lgoE rithm tht llows the simultion of distriution in whih the numer of vriles is itself rndom y produing n ergodi wrkov hinF he twgwg smpler onsists of two stepsX

• 1: e perturtion kernel is hosen from mixture of perturtion kernels nd the urrent stte of the wrkov hin is pertured using this kernelY

• 2: en eptne rtio is omputed nd deision is mde to either keep the wrkov hin in the urrent stte or evolve it to new stte fter the perturtionF he twgwg smpler is presented in elgorithm RF his smpler n e used to e'etively simulte mrked point proess modelsF xeverthelessD this e'etiveness omes t prieF he smpler requires high numer of itertions until onvergene nd usully results in very high omputtion timesF yn one hndD this omputE Algorithm 4 eversile jump wgwg smpler proposed y qreen IWWSF

IF snitilize X 0 = x 0 nd i = 0Y PF et itertion i with X i = xX
• ghoose kernel type m ∈ M ording to the proility p i (x) ertur x to on(gurtion x ording to he twgwg performs suessive perturtions to the urrent on(gurtionF st n e esily oserved tht suh proedure n tke lrge mount of time to ompleteD espeilly in the se of lrge senes with signi(nt numer of ojetsF erdié nd vfrge PHIP proposed to perform multiple perturtions in prllel y exploiting the onditionl independene of ojets outside the lol neighorhoodF husD they prtitioned the imge spe K into regulr mosi of independent ells using dtEdriven prtitioning shemeF erdie PHIQ demonstrtes tht the trnsition proility of two suessive perturtions tht our in independent ells equls the produt of the trnsition proilities of the individul perturtionsF wo ellsD c s nd c s re sid to e independent if nd only ifX

P m (x → •)Y gompute the qreen rtioX R = Q m (x → x) Q m (x → x ) π(dx ) π(dx) ghoose X i+1 = x with proility min(1, R) nd X i+1 = x
min p∈cs,p ∈c s ||p -p || ≥ ε + 2δ max , @QFUHA
where p nd p re pixels within the ells c s nd c s respetivelyD ε is the width of the neighoring reltionship etween ells nd δ max is the lrgest perturtion move llowedF sn other wordsD two ells re independent if the trnsition proility of rndom perturtion performed in one ell does not depend on the ojets onE tined in the seond ellF his property is visully depited in pigure QFRF he ells re then regrouped into 2 dim K sets suh tht no two neighoring ells elong to the sme setF his regrouping sheme is used to ensure the mutul inE dependene etween ells from the sme setD generilly lled mic-set @eFgF wuE tully sndependent gells setAF pigure QFS shows regulr prtition sheme on KD with dim K = 2 @leftA nd dim K = 3 @rightA respetivelyF fy using dtEdriven prtitioningD erdié nd vfrge PHIP sueeded in developing nturl nd e0E ient sheme for simulting nonEuniform point distriutionsF pigure QFT shows the di'erene etween regulr prtition sheme on given imge @with dim K = 2A @A nd dtEdriven prtitioning sheme for the sme imge @dAF pinllyD they formulted the generl trnsition kernel Q s mixture of uniform suEkernels Q c,m D eh kernel eing de(ned on ell c of KD the kernel type m ∈ M suh thtX

∀x ∈ Ω, Q(x → •) = c∈K m∈M q c,m Q c,m (x → •), @QFUIA
where q c,m is the proility of hoosing kernel Q m (x → •) in the ell cD given yX

q c,m = P (m) # cells in K . @QFUPA
he kernel de(ned in eqF QFUI is emedded into the wgwg dynmisF his smpler enles the exeution of prllel perturtions to intrinsilly nonEuniform point distriution sed on the dtEdriven prtitioning shemeF he smpling proedure is presented in elgorithm SF 56 Chapter 3. Using Point processes for object detection Algorithm 5 rllel smpler proposed y erdié nd vfrge PHIPD sed on the dtEprllel distriuted proessing modelF

IF snitilize X 0 = x 0 nd i = 0Y
PF gompute the dtEdriven spe prtitioning tree KY QF et itertion i with

X i = xX • ghoose miEset S mic ∈ K nd kernel type m ∈ M ording to the proility c∈S mic q c,m
• por eh ell c in S mic X ertur x in ell c to on(gurtion

x ording to Q c,m (x → •)Y gompute the qreen rtioX R = Q c,m (x → x) Q c,m (x → x ) π(dx ) π(dx) ghoose X i+1 = x with proility min(1, R) nd X i+1 = x otherE wiseY 3.3. Optimization 57
pigure QFRX gell independeneF veftX gell independene is not ensuredF he width of the ell is not lrge enough nd heneD perturtion of the ojet in ell c 1 depends on the ojet in ell c 3 Y ightX gell independene is ensuredF he width of the ell is lrge enough so tht ny perturtion of n ojet in ell c 1 would not depend on n ojet in ell c 3 F smge ourtesy of erdie PHIQF pigure QFSX egulr prtition sheme on KD with dim K = 2 @leftA nd dim K = 3 @rightAF smge ourtesy of erdie PHIQF pigure QFTX @A en imge of otsY @A he lss of interest is estimted from the input imge @for exmpleD irth mp n e used to otin this initil estimE tionAY @A he orresponding proilities q c,i D when regulr prtitioning sheme is ppliedY @dA he orresponding proilities q c,i when dtEdriven prtitioning sheme is ppliedF 58 Chapter 3. Using Point processes for object detection

Multiple birth and death algorithm

e di'erent ttempt to simulte point proesses is the multiple irth nd deth @wfhA lgorithm proposed y hesomes et lF PHHWF he ide is tht t eh itertion iD new rndom on(gurtion x @multiple ojetsA is dded nd then nonE (tting ojets re removed from x = x i ∪x F he wfh lgorithm hs een designed s ontinuousEtime reversile proessD whih is then disretized using wrkov hinF he uthors show tht this pproximtion gurntees wek onvergene to the desired distriution πF he lgorithm is presented in elgorithm TD where s efore Λ is the intensity of the underlying oisson proess nd T is the temperture prmeter used for the simulted nneling shemeD while α Λ nd α T re prmeters used to derese the intensity of the oisson proess nd the temperture respetivelyF Algorithm 6 wultiple irth nd deth smpler proposed y hesomes et lF PHHWF

IF snitilize X 0 = x 0 nd i = 0Y PF Λ = Λ 0 nd T = T 0 QF epetX • firthEstepX generte x D reliztion of oisson proess of intensity Λ • x = x i ∪ x • hethEstepX por eh x i ∈ xD ompute a T (x i ) = exp U (x\x i )-U (x) T D nd drw p from uniform distriution • sf p < Λa T (x i ) 1+Λa T (x i ) then remove xY x ← x \ x • et x i+1 = xD Λ i+1 = Λ i a Λ D T i+1 = T i a T D i = i + 1 nd go to firthEstepF
e modi(ed version of the wfh is the multiple irth nd ut @wfgA lgorithm developed y qmlEildin et lF PHIHF wfg proposes to use the grphEut lgoE rithm for energy minimiztionF ih ojet in the on(gurtion represents node in the grphD while the edge weights re ssigned sed on the energy of eh oE jetF henD the minEut mxE)ow lgorithm is used to (nd the minimum ut tht mximizes the )ow within the grph nd thusD minimizes the energy funtionF wore detils n e found in qmlEildin et lFD PHIHD qml ildin et lFD PHIIF

Jump-diusion processes

trting with the seminl pper of werton IWUT jumpEdi'usion proesses hve een intensively studied in the demi (nne ommunityF sn omputer visionD these proesses eme known through the work of qrennder nd willer IWWR on uilding lgorithms for utomti hypothesis formtion nd ws suessfully pplied fterwrd in pplitions suh s trget trking rivstv et lFD IWWSD PHHPD imge segmenttion rn et lFD PHHR or more reently in texture nlysis vfrge et lFD PHIH nd 3h stereo reonstrution vfrge et lFD PHIHF he jumpEdi'usion smpler omines the wgwg rstingsD IWUHD qreenD IWWS nd the vngevin equtions qemn nd rungD IWVTF ih dynmi plys di'erent role within the smplerX the jump dynmis re used to perform reversile dimensionl jumps etween the suspes of ΩD with Ω = n∈N Ω n D wheres the di'usion dynmis re used to ondut stohsti di'usion within eh ontinuous suspe Ω n driven y frownin motionsF 3.3.3.1 Jump process he jump proess is wgwg lgorithmD generlly twgwg lgorithm heE somes et lFD PHIIF eversile jumps re performed etween the di'erent suE spes of Ω ording to the irth nd deth kernels disussed in etion QFQFIFRF vol perturtions re no longer neessry s they re sustituted y the di'usion proess tht llows for n e0ient explortion of the suspeF 3.3.3.2 Diusion process he di'usion proess is used to explore the suspes ontinuouslyF sn ontrst to the wetropolisErstingsEqreen lgorithmD these lgorithms re hrterized y the sene of rejetionF he dynmis through whih the urrent on(gurtion evolves t every itertion n e interpreted s stohsti grdient desentF he di'usion proess is sed on the vngevin equtions whih we will further detil in this seE tionF vet X e point proess on W nd suppose tht X follows s qis distriution suh tht f (X) = 1 c exp -U (X) F en lterntive to the wetropolisErstingsEqreen lgorithm is to de(ne di'usion eqution whih onverges towrd the trget distriE ution π(dX)F he following stohsti di'erentil eqution gives suh dynmisX

dX t = -∇U (X t )dt + W t , @QFUQA
where X t D t ∈ R + nd W t is frownin motionF he di'usion eqution hs to e disretized in order to e simultedF hese dynmE is re known s the vngevin dynmisF vet τ (δ) = {τ i , i = 0, t} e disretiztion of the intervl [0, t] y the time steps δ i = τ i+1 -τ i D then the disrete pproximtion of the proess X n e written ∀u ∈ W sX

z u (0) = x u (0) z u (i + 1) = z u (i) + a u (Z(i))δ i + W u (τ i+1 ) -W u (τ i ), @QFURA where a u (Z(i)) = -∇ u U (Z(i))F e
h i (x) ∝ h 1 T i (x), @QFUTA
where T i is sequene of tempertures tht tends towrds 0 s i goes to in(nityF imulted nneling hs een proven to onverge for reversile jump wgwg lE gorithm y toi et lF PHHS nd for irth nd deth proesses y vn vieshout IWWRF roweverD the hoie of the temperture derese funtion hs signi(nt impt on the qulity of the simulted nnelingF his temperture prmeter is ontrolled using ooling sheduleF he ooling shedule is omposed of four stgesX IF The temperature is increasedF es resultD the density eomes lose to uniform distriutionF his step ensures tht the initil temperture is dE equte for the given density funtionF sf the strting temperture is too lowD the proess will most likely end up in lol minimumF sf the strting temperE ture is too highD the onvergene is not 'etedD howeverD the omputtion ost inresesY PF The temperature decreases slowlyF huring this step the proess explores the di'erent modes of the density nd eomes more restritive with the numE er of itertionsF sf the temperture derese is slow enoughD the proess will identify one of the glol mxim of the density funtionY 3.3. Optimization 61 QF The temperature is near zeroF huring this step smll djustments re mde to the on(gurtion of ojetsF he density (nlly rehes one of the glol optimY RF Convergence is reachedF he stopping ondition hs een ttinedF sn generlD the ooling shedule stops when the energy of the proess hs not evolved for ertin numer of itertions or when ertin temperture is rehedF st n esily e oserved tht the hoie of the temperture derese funtion is ruil for rehing the glol mximumF he most populr derese funtions re disussed elowF 3.3.4.1 Logarithmic decrease he logrithmi derese given y

T i = C log(i + 1) @QFUUA
hs een proven in theory to ensure the onvergene towrds the glol optimum when the onstnt C is lrger thn the deepest lol mximumF roweverD in prtie it is impossile to foresee the deepest lol mximum nd the derese of the temE perture is very slow whih renders this shedule imprtil for implementtion purposesF snstedD geometri derese funtion is usully preferred in prtieF xote tht in this se howeverD the onvergene to the glol optimum is no longer gurnteedF 3.3.4.2 Geometric decrease he geometri derese is given y pigure RFPX @A gloseEup on n re with neighoring ots within the hrorY @A e typil exmple of ot where the pilothouse nd superstrutures pper s drk spots within the otY @A sn redX the exterior order F ρ (u) used to ompute the ontrst distne to the interior of the ellipseY @dA sn redX the exterior order F ρ (u) nd in greenX the interior order I 1 (u)F used y o et lF PHIH to jointly detet nd estimte multiple ojets from imge oservtionsF e qis proess is used to model the density funtion f whih is de(ned s followsX

T i+1 = T i if i mod N = 0 αT i otherwiseD @QFUVA where 0 < α < 1 is oe0ient tht is used
W = K × M = [0, IW M ] × [0, IH M ] × [a m , a M ] × [b m , b M ] × [0, π]. @RFIA rereD IW M nd IH M represent
f θ (X = x|y) = 1 c(θ|y) exp -U θ (x,y) @RFPA with c(θ|y) = Ω exp -U θ (x,y) µ(dx) @RFQA
eing the normlizing onstntF he model is desried in terms of the energy funtion U θ (x, y)D whih is omposed of two termsX n externl energy term whih re)ets how good the model (ts the tul dtD denoted U ext θext (x, y)D nd n internl energy term inluding onstrints imposed on the on(gurtionD denoted U int θ int (x)F he vetor θ ext ontins the pE rmeters of the externl energy term tht hve to e estimtedD while θ int ontins the prmeters of the internl energy termF reneD the prmeter vetor for the entire model n e written s θ = {θ ext , θ int }F 4.1.1 External energy term sn this pprohD the omputtion of the externl energy term tkes ple lollyD for eh ellipseF he externl energy of the entire on(gurtion is omputed s the sum of the individul externl energiesX

U ext θext (x, y) = γ cnt u∈x U ext (u, y), @RFRA harbors
pigure RFQX veftXisul representtion of the qulity funtion used to onstrut the externl term in eqution RFSF he more n ellipse from on(gurtion x overlps n ojet in the imgeD the lower the vlue of the qulity funtion nd thusD the lower the externl energy termF sf n ellipse does not overlp with n ojet in the imgeD the qulity funtion returns high vlue whih in turn leds to higher vlue for the externl termD mking the ellipse less prole to e prt of the true on(gurtionF ightX isul representtion of the qulity funtion wFrFtF its rgumentF where γ cnt represents the externl energy weight nd hs to e estimtedF he omputtion of the lol externl energy termD U ext (u, y)D relies on the use of ontrst distne mesure d B (•, •) etween the interior of the ellipseD denoted y uD nd two prede(ned ordersD s shown in pigure RFPF he externl orderD F ρ (u)D is used to detet the ojet with respet to the kgroundF etD when seen from oveD most ots hve drk spot where the pilothouse or superstrutures re lotedF his leds to lower ontrst distne nd thus lower detetion proilityF hereforeD n interior orderD I 1 (u)D is lso onsideredF he e'ets of omputing the distne etween the interior nd externl orders re twofoldX (rstD the overll detetion proility inreses when this distne is highY nd seondD the flse lrm rte deresesF he externl term eomes thereforeX

U ext (u, y) = Q d B (u, F ρ (u)) d 0 (y) + γ c Q d B (I 1 (u), F ρ (u)) d 0 (y) , @RFSA
where γ c is the weight of the ontrst mesure omputed using the interior orderF he ontrst distne mesureD d B (•, •)D is similr to the fhtthryy distne @see qoudil et lF PHHRAX

d B (p, q) =   (µ p -µ q ) 2 4 σ 2 p + σ 2 q - 1 2 log   2 σ 2 p σ 2 q σ 2 p + σ 2 q     @RFTA
where (µ • , σ 2 • ) represent empiril mens nd vrinesF he threshold d 0 (y) for the ontrst is determined sed on the imgeD y nd will

Q(x) = 1 -x 1/3 if x < 1 exp(-x-1 3 ) -1 if x ≥ 1 @RFUA
is qulity funtionD depited in pigure RFQD tht is de(ned on Q : R + [-1, 1]F he lower the energy of one on(gurtionD the etter the on(gurtion (ts the imgeF husD the qulity funtion ttriutes negtive vlue to well pled ojets @eFgF those ojets u for whih d B (•, •) is higher thn the threshold d 0 (y)A nd positive vlue to mispled ojetsF he use of the ui root llows for moderte penliztion when the output of the distne mesure is ner the thresholdF 4.1.2 Internal energy term he internl energy term is deomposed into two prts tht inorporte the three ssumptions mde in the eginning of this hpterF he (rst prt tkes re of overlppingD mening tht ojets re not llowed to overlp more thn given extentF he seond prt hndles the onstrints relted to the orienttion nd lignment of the ellipsesF he lol orienttion of ots is determined sed on imge ues nd ellipses whih hve orienttions lose to the expeted one re fvoredF woreoverD on(gurtions where neighoring ojets hve similr orienttions re lso fvoredF 4.1.2.1 Non-overlap constraint he (rst prt of the internl energy term orresponds to penliztion of overlpE ping ellipsesD voiding the detetion of the sme ojet severl timesF he proposed model uses hard core proess to hndle ojet overlppingD mening tht ll on(gE urtions ontining ellipses tht overlp more thn given extent will e disregrded 68 Chapter 4. Spatial marked point process model for boat extraction in harbors @iFeF the energy vlue ssigned to suh on(gurtions will e in(nitely highAF reneD denoting y A(u i , u j ) =

Area(u i ∩u j ) min(Area(u i ),Area(u j )) the re of intersetion etween the ojets u i nd u j D the nonEoverlpping energy term n e de(ned sX

U int o (x) = 1≤i =j≤n(x) t s (u i , u j ) @RFVA withX t s (u i , u j ) = 0 if A(u i , u j ) < s +∞ otherwise @RFWA
where s ∈ [0, 1] orresponds to the mount of overlpping llowed y the model nd n(x) is the numer of ojets in the on(gurtion xF e visul representtion of the nonEoverlpping onstrint is presented in pigure RFRF eordinglyD ll on(gurtions ontining t lest two ojets tht overlp to higher rtio thn spei(ed y s re prohiitedF 4.1.2.2 Determining the local orientation of the objects e method proposed y vi nd friggs PHHW for rod extrtion in high resolution stellite imges of urn resD is used in order to lolly determine the orienttion of the doks using the wterF he ide is to pply n edge detetor nd then identify longD stright lines using the rough trnsform sed on the edge detetor responseF he orienttion of the lines n then e esily determinedF he gnnyEherihe edge detetor whih ws initilly proposed y gnny IWVT nd then extended y herihe IWVU is used to seprte the wter re from the other strutures in the imgeF ine ll pixels in the wter re hve similr vlues in the sene of high wvesD most edges re due to the high ontrst etween the wter re nd the lnd strutures or otsF eny edge detetor n e ppliedF he gnnyEherihe edge detetor ws seleted for its moderte edge outputF foth high oversegmenttion or undersegmenttion leds to lower ury in the following stepsF vi nd friggs PHHW desried two key oneptsX reference circles nd central pixelsF por eh pixel pD its referene irle C(p) is the irle with the lrgest rdiusD entered t pD tht does not ontin edge pointsF sn other wordsD the rdius of the referene irle is the mximum distne from the pixel to the losest edge pointF e pixel is onsidered entrl pixelD if it hs the lrgest referene irle mong the neighoring pixelsF hese onepts re illustrted in pigure RFSF he ide ehind these onepts is tht ll the entrl pixels form the enter line of the wter reF he inomplete or inurte edges will slightly 'et the lotion of the entrl pixelsD ut the method hs high tolerne over suh errorF he entrl pixels re found using the distne trnsform sed on the result of the edge detetionF he entrl pixels re plotted onto inry imgeD where pixel vlue of 1 signls the presene of entrl pixel t tht lotion nd pixel vlue of 0 signls the sene of entrl pixelF efter extrting the entrl pixelsD rough trnsform is pplied to detet the lines in the resulted inry imge nd ompute their orienttionF e visul desription of the pigure RFSX isul representtion of the onepts of reference circles nd center pixels used to determine the lol orienttions of ojets in the seneF method is presented in pigure RFTF husD the orienttion of the wter re hs een estlished lolly nd set of relevnt pixels in the imge hs een otinedF ih suh pixel ontins informtion out the orienttion of the wter in tht reF

Alignment and orientation constraints

es stted in the eginning of this hpterD neighoring ots should hve similr orienttionsF husD n lignment intertion etween two neighoring ellipses u 1 nd u 2 is de(ned in the following wyX

u 1 ∼ al u 2 ⇔ d ω (u 1 , u 2 ) ≤ d ωmax d C (u 1 , u 2 ) ≤ d Cmax @RFIHA where d ω (u 1 , u 2 ) = |ω 1 -ω 2 |
is the di'erene of the orienttions of the two ellipses u 1 nd u 2 nd d ωmax is the mximum ngle llowed etween two neighoring ojetsD while

d C (u 1 , u 2 ) = |d(c 1 , c 2 ) -(b 1 + b 2 )|D where d(c 1 , c
2 ) stnds for the iuliden distne etween the enters of the ellipses nd d Cmax is the mximum distne llowedF henD n internl energy term tht promotes this lignment is designed in the following wyX

U al (u 1 , u 2 ) =    δ (d ω (u 1 , u 2 ), d ωmax ) if u 1 ∼ al u 2 0 otherwise @RFIIA
where (x, x max ) is rewrd funtion introdued y yrtner et lF PHHVD tht fvors lignment nd is de(ned sX

(x, x max ) = - 1 x 2 max 1 + x 2 max 1 + x 2 -1 , f or x ≤ x max .
@RFIPA sn the previous susetionD the lol orienttion of ojets in sene hs een deterE mined sed on the method proposed y vi nd friggs PHHWF his predetermined harbors pigure RFTX orkE)ow of the lol orienttion detetion methodF opEleftX snitil imgeF opErightX isuliztion of the referene irlesF he righter the pixel vluesD the lrger the rdius of the referene irleF fottomErightX genter pixel extrtion sed on the rdius of the referene irlesF fottomEleftX vines otined using the rough trnsformF he orienttion of these lines is used to determine the lol orienttion of the ojetsF orienttion n e used to fvor ellipses whih tend to lign with itF qiven n ellipse uD the nerest relevnt pixel in its sptil neighorhood is identi(edF sf no relevnt pixel is found within ertin distneD ellipses tht hve similr orienttions to other ellipses in their neighorhood re fvoredF sf relevnt pixel is foundD its poE sition wFrFtF the semiExes of the ellipse is determined nd ellipses perpendiulr to the orienttion indited y the relevnt pixel re fvoredF husD the (nl internl energy term tht refers to oth orienttion nd lignment eomesX

U int alω l (x) = u∈x U int alω l (u) @RFIQA whereX U int alω l (u) =    0 if relevnt pixel is found nd ||ω u -ω l || > d ωmax γ al v∈x U al (u, v) otherwise @RFIRA
where ω u is the orienttion of ojet u nd ω l is the orienttion perpendiulr to the one retined in the relevnt pixelF 4.1.2.4 Total internal term he totl internl term is the sum of its two prts nd is therefore written sX

U int θ int (x) = U int o (x) + U int alω l (x) @RFISA
with θ int = {γ al }D the weight of the lignment intertionD present in eqution RFIRF

Total energy term

he totl energy term is written s the sum of the externl energy nd internl energy termsD s followsX • γ c is the weight of the ontrst distne etween the interior order nd the exterior order wFrFtF the ontrst distne etween the interior of the ellipse nd the exterior orderY

U θ (x, y) = U ext θext (x, y) + U int θ int (x) = γ cnt u∈x U ext (u, y) + U int o (x) + U int
• d 0 is the threshold used in the qulity funtion Q(•) to ssess the ontrst distneY

• ρ is the width of the exterior order of n ellipseY

• s ∈ [0, 1] is the llowed overlpping rtioY
• γ al is the weight of the lignment term with respet to the overlpping term within the internl energyF istimting utomtilly the entire prmeter vetor θ is di0ult tskF hereforeD the prmeters tht hve to e estimted need to e nrrowed downF sndeedD the prmeters ρ nd s n e esily set empirilly s they hve stright forwrd physil interprettionF es suhD ρ is set to ρ = 2F his vlue n e esily motiE vtedX if ρ = 1D the numer of pixels tht mke up the order is smllD resulting 72 Chapter 4. Spatial marked point process model for boat extraction in harbors in poor estimtes of the men nd the vrine of the orderD when omputing the externl energy termY if ρ ≥ 3D the order will ontin pixels from the neighoring ojetsD whih will led to higher men vlue of the order nd thusD lower ontrst distneF e visul representtion of the order width ρ is shown in pigure RFUF he overlpping rtio s is set to s = 0.1D mening tht only on(gurtions with ojets overlpping less thn 10% re llowedF ellowing smll mount of overE lpping is ruil for deteting tightly lustered ojetsD s n e seen in pigure RFUF

pigure RFUX e visul representtion of the order width ρ for few seleted ots of di'erent sizesF @A he input imgeF @A ρ = 1F his is good vlue for very smll otsD howeverD it does not provide su0ient numer of pixels to ompute the relevnt sttistisF @A ρ = 2F his is the est vlue tht (ts oth smll nd lrge otsF wen nd vrine of the order pixels n e omputed for smll ots s wellF @dA ρ = 3F sn this seD the order signi(ntly overlps with neighoring ojets @oth ots nd doksA nd the ontrst vlue drops ordinglyF por lrge ojetsD the dditionl pixel in the order width does not led to signi(nt hnges in the order sttistisF he other prmeters however re more di0ult to set y hndF xeverthelessD trying to estimte ll remining prmeters with iwElike methods results in high omputtionl omplexity nd low onvergene propertiesF hereforeD the weights γ cnt nd γ al re estimted using the iw lgorithm presented in hpter QD while heuristis re devised for the prmeters d 0 nd γ c F 4.2.1 Determining the weights γ cnt and γ al he min di0ulty in estimting these prmeters drws from the ft tht oth the on(gurtion of ojets x nd the mrginl density of the oservtions f θ (y) re unknownF sn suh situtionsD iwElike lgorithms re known to e e0ientF his type of itertive lgorithms provide n estimte of θ sed on the serh for lol mximum of the imge likelihood f θ (y)F xeverthelessD the iw lgorithm is intrtle for the proposed modelF husD stohsti vrint of the iwD lled iw geleux nd hieoltD IWVS @see ghpter QAD is n pproprite solution in this seF xeverthelessD it is not possile to otin trtle expression of the expettion sine the normlizing onstnt in eqF RFQ is itself intrtleF husD the likelihood is pproximted using the pseudoElikelihood de(ned in eqF QFPID where the extended pngelou intensity λ θ (u; x, y) ssoited with ojet u is written sX

λ θ (u; x, y) = β exp   -γ cnt U ext (u, y) - v∈x,v =u t s (u, v) -γ al v∈x,v =u U al (u, v)   .
@RFIUA he initiliztion step is very importnt for the onvergene of the iw to good solution oert nd gsellD PHHSF por the proposed modelD γ cnt nd γ al re initilized suh tht the ojets in the (rst on(gurtion re independentF wore preiselyD γ 0 cnt nd γ 0 al re unique nonEzero roots of W λ θ (u; ∅, y)Λ(du) -βD where λ θ (u; ∅, y) is the pngelou intensity ssoited with the empty on(gurtion x = ∅F impiril tests hve shown tht γ 0 al should hve lower vlue thn γ 0 cnt F 4.2.2 Determining the weight γ c he weight γ c determines the in)uene of the ontrst distne etween the interior order nd the exterior order within the totl dt energy termF es shown in pigure RFPD drk spot usully ppers in the middle or lower prt of the ojetD where the pilothouse or superstrutures reF his drk spot dereses the ontrst distne etween the exterior order nd the interior of the ellipseF hereforeD n interior orderD I 1 (•) of (xed width equl to 1 is lso onsideredF qiven the smll width of I 1 (•)D generlly only few pixels mke up the interior orderF husD the min role of this term is to shift the lne in ses whereD given n ojet u nd its order

F ρ (u)D the ontrst distne d B (u, F ρ (u)) is lose to the threshold d 0 F ril nd error experiments hve shown tht for γ c ≤ 0.6D Q d B (I 1 (u),F ρ (u))
= avg(C 1 ) nd m 2 = avg(C 2 )Y RF he threshold t itertion i is omputed s the rithmeti men of the two vergesX D i = (m 1 + m 2 )/2Y
SF epet steps 2 E 4 using the newly omputed threshold until it onverges @D i+1 = D i AF he (nl threshold is otined using the formulX d 0 = (m 1 /m 2 ) * (g max /(g max -D i ))D where g max is the mximum greyElevel vlueF

Optimization

e mrked point proess is fully de(ned y its unnormlized density f (x) wFrFtF to referene mesureD whih in this model is the homogeneous oisson mesure toyn nd toynD IWWRF imulting point proesses is usully done using the reversile jump wrkov hin wonte grlo @twgwgA smpler qreenD IWWSD presented in hpter QF twgwg simultes disrete wrkov hin (X i ) i∈N on the on(gurE tion spe Ω tht onverges towrds n invrint mesure spei(ed y the energy U of the modelF et eh stepD the urrent on(gurtion x of the hin is pertured to new on(gurtion x ording to trnsition kernel {Q m (x, •)} m∈M F he new on(gurtion x is epted with ertin proilityD lled the qreen rtioF por etter resultsD the twgwg is usully emedded into simulted nneling sheme hesomes et lFD PHII nd thusD the qreen rtio is written sX

R = Q m (x → x ) Q m (x → x) exp U (x) -U (x ) T i , @RFIVA
where T i is relxtion prmeterD lso lled tempertureF he stndrd trnsiE tion kernels employed to simulte the proposed model reX ojet retionGremovl @generlly known s the firth nd heth kernelA nd lol perturtion kernels @eFgF rottionD trnsltion nd sleAF he e0ieny of the optimiztion proedure depends on severl prmeters suh s the size of the serh spe KD the size of the ojet spe WD the distriution nd numer of ojets in the seneD the smpler usedD etF he size of the ojet spe W is predetermined y the model nd the distriution nd numer of ojets is onsidered to e unknown yrtnerD PHHRF roweverD the size of the serh spe in the prtiulr se of ot detetion n e redued y using the simple heuristi tht ots re expeted to pper only in wter resF purthermoreD prllel imE plementtion of the twgwg smpler hs lredy een introdued in hpter QF sn this setionD we desrie the perturtion kernels used to simulte the mrked point proess model presented in etion RFIF henD the drwks of using the prllel implementtion devised y erdié nd vfrge PHIP re illustrted nd n lterntive multipleEore implementtion tht ddresses the identi(ed drwks is desriedF pinllyD wter G lnd disrimintion lgorithm to redue the serh spe K is presentedF 4.3.1 Perturbation kernels used for object detection gonsider the mrked point proess model presented in this hpterF he ojet spe is

W = K × M ⊂ R 2 × R 3 nd n ojet u is given y u = (x u , y u , a u , b u , ω u )D
where (x u , y u ) is the position of the ojet nd (a u , b u , ω u ) re the semiEmjor xisD semiEminor xis nd orienttion of the ellipseF o simulte the model presented in etion RFID we hve used the following perturtion kernelsX

• Uniform Birth and DeathF his kernel is used to rete or delete ojets from the on(gurtion nd thusD hnges the dimension of the on(gurtion speF his stndrd kernel is mixture of two suEkernelsD one whih retes ojets nd nother whih deletes ojets from the on(gurtionF his kernel ssumes uniform genertion of ojets in W D ording to the proility lw ν(•) ν(W ) where ν(W ) is mesure on W D nd uniform removl of ojets from the urrent on(gurtionF he kernel n e written sX

Q BD (x, •) = p b (x)Q B (x, •) + p d (x)Q D (x, •), @RFIWA
where we tke

p b = p d = 0.5 in our experimentsD Q B (•, •) is the irth suEkernel nd Q D (•, •)
is the deth suEkernelF he qreen rtios for these suEkernels n e written s followsX

BirthX he kernel proposes new on(gurtion x ∪ u nd the ssoited qreen rtio is given yX

R B (x, x ∪ u) = h(x ∪ u) h(x) p d p b ν(W ) n(x
R D (x, x \ u) = h(x \ u) h(x) p b p d n(x) ν(W ) . @RFPIA
• Local perturbationsX st is more e0ient to perform smll perturtions on n existing ojet rther thn propose dethD immeditely followed y irthF vol perturtions re spei(lly designed to hieve this e0ienyF e use symmetri perturtions in order to gurntee the detiled lne onditionF vet T = {T a : a ∈ A} e fmily of symmetri perturtions prmetrized y aF he perturtion kernel ssoited to this fmily onsists in hoosing n ojet u uniformly nd t rndom from the on(gurtion x nd proposing perturtion y pplying T a to u suh tht v = T a (u)F qiven tht oth the perturtion T a nd the ojet u re hosen rndomlyD we n write the qreen rtio for this kernel sX

R L (x, (x \ u) ∪ v) = h((x \ u) ∪ v) h(x) .
@RFPPA e hve used three types of symmetri perturtions in our experimentsX trnsltionD rottion nd sleF he fmily of translations is de(ned in [-δ x , δ x ] × [-δ y , δ y ] nd trnsition T [dx,dy trnsltes the enter (x u , y u ) of the ellipse u to new positionD provided tht this position is still in

K = [0, I hmax ] × [0, I wmax ]X T [dx,dy]        x u y u a u b u ω u        =        (x u + d x )[I hmax ] (y u + d y )[I wmax ] a u b u ω u        . @RFPQA e rotation is prmetrized y vetor [d ω ]D with d ω ∈ [-δ ω , δ ω ]F his perE
turtion hnges the orienttion of n ellipse uX • Heterogeneous models hve een reently developed due to the pperne of heterogeneous systems @iFeF systems tht hve one or more host gs nd one or more qsAF ghe is suh prllel progrmming model developed y xshse xidiD PHIQ nd designed to develop pplitions tht sle with the inresing numer of ores provided y qsF he prllel system onsists of host @iFeF gA nd omputtion resoure @iFeF qA nd the tsks re performed on the q through numer of thredsF he min dvntge of using the q is its mssively prllel strutureF es ghe is spei(lly developed for xshse grphil rdsD the ommunity worked on produing royltyEfree stndrdD known s ypengv uhronosD PHIQD whih is designed for generl purpose prllel omputing ross gsD qs nd other proessorsF ypengv distinguishes etween devies @minly gs or qsA nd the host @gA nd the key ide ehind this model is to write kernels @eFgF funtions tht run on ypengv deviesA nd pplition progrmE ming interfes @essA for reting nd mnging these kernels whih re then ompiled for the trgeted devieF erdié nd vfrge PHIP use ghe for their q implementtionF hey dedite thred to eh simultneous perturtion nd show tht the more ells in the prtition treeD the more e0ient the smplerF hey lso optimize the implementtion through memory olesing @iFeF omining multiple memory esses into single trnstionA nd ojet indexingF sn terms of omputtionl e0ienyD the ommunition osts of loding the dt into memory ounts for lrge mount of timeF hen hndling very lrge dt setsD the memory trnsfer time etween the g nd the q n e very longD espeilly when repeted trnsfers hve to e mdeF e multiEore implementtion does not enounter this lgF purthermoreD portility is still hevily deted issue for q implementtionsF reneD we opted for multiEore implementtion of the prllel smplerF 4.3.2.2 Parallel implementation of RJMCMC using multiple cores with shared memory e solution to hndle split ojets t ell oundry is not ovious if only the on(gE urtion within ell is onsideredF roweverD if the on(gurtions of ojets within the neighoring ells re lso onsidered suh errors would e esily solvleF snE deedD shred memory model n e e0iently used to tkle this prolemF et eh itertionD for eh ell in the hosen miEset S mic D the on(gurtions from the neighoring ells re tken into ount when performing perturtionsD s n e oserved in eqF RFPTF xote tht this does not rise synhroniztion issues etween omputing units when essing the shred memoryD sine the omputing units only red the informtion in the neighoring ells nd do not modify itF he smpler we proposed is desried in elgorithm UF wo dditionl steps re dded to the smpler to mke it spei( to the prolem of ot detetionX IF e preEproessing step is inluded into the smpler tht omputes the wter mskF e will disuss this step in detil in etion RFQFQY PF he spe prtitioning tree is pruned sed on the wter msk @step 3AF

T [dω]        x u y u a u b u ω u        =        x u y u a u b u (ω u + d ω )[π]        . @RFPRA pinllyD scale is prmetrized y vetor [d a , d b ] with d a ∈ [-δ a , δ a ] nd d b ∈ [-δ b ,
T [da,d b ]        x u y u a u b u ω u        =        x u y u (a min + (a u -a min + d a ))[a max -a min ] (b min + (b u -b min + d b ))[b max -b min ] ω u        . @
PF snitilize X 0 = x 0 nd i = 0Y
QF gompute the dtEdriven spe prtitioning tree K nd trunte it sed on the wter mskY RF et itertion i with X i = xX

• ghoose miEset S mic ∈ K nd kernel type m ∈ M ording to the proility c∈S mic q c,m

• por eh ell c in S mic X ertur x in ell c to on(gurtion x ording to Q c,m (x → •)Y etrieve the on(gurtion z from the neighoring ellsY gompute the qreen rtioX

R = Q c,γ (x ∪ z → x ∪ z) Q c,γ (x ∪ z → x ∪ z) exp U (x ∪ z) -U (x ∪ z) T t @RFPTA ghoose X i+1 = x with proility min(1, R)D else X i+1 = xY
• pdte T i+1 = αT i @in our tests α = 0.95AF

Water / Land discrimination

fots re prtiulr type of ojet tht only pper in the presene of wter @with the exeption of dry dosD whih re not of interest hereAF husD when presented with lrge stellite imgeD the (rst step in serhing for ots is to restrit the serh to wter resF he wter re n e generlly identi(ed s lrge re of low rdiometri vluesF roweverD simple threshold is not su0ientF ll uildings usully st shdows onto the groundD whih pper s drk res s shown in pigure RFW @topAF pigure RFW @middleA shows the result when pplying thresholdD sed on the ytsu method @ytsuD IWUWAF he flse lrm rte used y shdows is very highF ghoosing size s ritil feture to distinguish etween wter nd shdows is not su0ientD sine rivers nd lkes n e esily missedF sndeedD the interest lies in extrting the wter res under the ondition tht the numer of missed detetions should e zero nd the numer of flse lrms should e minimlF he di0ulty of this prolem lies in (nding pproprite fetures to do the seprtionF Chapter 4. Spatial marked point process model for boat extraction in harbors @A @A @A @dA @eA @fA pigure RFWX @AD @A smge of ots inside nd outside hror eirus h8Y @AD@dA terGvnd disrimintion results fter pplying threshold to the input imgeF righ flse lrm rte due to shdowsY @eAD@fA terGvnd disrimintion results using the lgorithm desried in elgorithm VF le RFPX untittive nlysis of the proposed ot extrtion lgorithm for pot 5 imgesF he detetion ury dereses s the informtion on the orienttion of the ots eomes less relileF his is prtiulrly true in pigure RFIH @ottomA where the orienttion of the ots in the urved re is prtilly nonEexistentF 84 Chapter 4. Spatial marked point process model for boat extraction in harbors @A @A @A pigure RFIHX @A pot 5 imges of ots in hror on the prenh ost gxiY @A rough lines otined for the extrtion of lol orienttion informtion for the otsY @A fot extrtion resultsF e totl of 501 @topAD 169 @seond rowAD 427 @third rowA nd 96 @ottomA ots re deteted respetivelyF 4.4. Results

85

@A @A @A pigure RFIIX @A leides imge of ots in hror in welourneD eustrli eirus h8Y @A rough lines otined for the extrtion of lol orienttion informtion for the otsY @A fot extrtion resultsF e totl of 31 @topA nd 39 @ottomA ots re deteted respetivelyF le RFQX untittive nlysis of the omputtionl e0ieny for ot extrtion on pot 5 imgesF 4.4.2 Computational eciency sn setion RFQD two wys hve een disussed whih inrese the omputtionl e0ieny of the proposed lgorithmX omputing wter msk to redue the serh spe nd using prllel implementtion of the twgwg smpler to perform multiple independent perturtions t the sme timeF rereD the omputtionl e0ieny of the model presented in this hpter is nlyzedF les RFQ nd RFS summrize the results wFrFtF the omputtionl e0ienyF he results show signi(nt derese in omputtion times for imges ontining vst lnd res when the wter msk is usedF he resons ehind rely on the ft tht the usge of wter msk redues the serh speD in some imges up to more thn 50%F his e'et is not visile in pigure RFIH @topA nd @seond rowA euse the lnd re is prtilly nonexistentF he sustined derese in omputtionl times with the numer of proessors shows tht the proposed optimiztion sheme is dpted to prllel implementtionF xote howeverD tht the speedEup is less signi(nt in imges with redued size @iFeF the imges in pigure RFIHAF he underlying ide ehind the prllel implementtion is speEprtitioningF he smller the imgesD the less ells re in the prtitioning tree nd heneD smller numer of ores n tully perform prllel omputtions while mintining the ellEindependene onditions outlined in setion RFQFPF From object detection to object tracking using a spatio-temporal marked point process model

ω ∈] -π 2 , π 2 ]
is the orienttion of the ellipse nd l ∈ [0, L] is its lelF husD n ellipse u n e de(ned s u = (c h , c w , t, a, b, ω, l) nd mrked point proess of ellipses X is point proess on W = K × M F hile the semiE xes a nd b nd the orienttion ω desrie the physil properties of n ellipseD the lel l is used s n identi(erF yjets with the sme lel ross the imge sequene form trkF sndividul trjetories re extrted y grouping ojets ording to their lelF sn the frmework of stohsti geometryD the ide of using lels to distinguish etween di'erent trjetories hs lredy een proposed nd implemented y o nd o PHIQD u et lF PHIR nd o et lF PHIRF e point proess distriution n e de(ned y its proility density funtion where the oisson point proess plys the nlogue role of the veesgue mesure on R d D with d eing the dimension of the ojet speF he qis fmily of proesses is used to de(ne the proility density s followsX

f θ (X = X|Y) = 1 c(θ|Y) exp -U θ (X,Y) @SFIA whereX • X = {x 1 ∪ x 2 ∪ • • • ∪ x t ∪ • • • ∪ x T }
is the on(gurtion of ellipsesD with x t eing the on(gurtion of ellipses t time tY

• Y represents the 3h imge ueY

• θ is the prmeter vetorY

• c(θ|Y) = Ω exp -U θ (X,Y) µ(dX)
is the normlizing onstntD with Ω eing the on(gurtion spe nd µ(•) eing the intensity mesure of the referene oisson proessY

• U θ (X, Y) is the energy termF he solution eing sought for is the on(gurtion whih mximizes the density f θ (X)D eFgF the most likely on(gurtion of ojets orresponds to the glol miniE mum of the energyX

X ∈ arg max X∈Ω f θ (X = X|Y) = arg min X∈Ω [U θ (X, Y)]. @SFPA
his on(gurtion is otined using the frmework of simulted nneling whih results in itertively simulting from the distriution f (X)

1 T dπ(X)D where the temE perture prmeter T is slowly deresing to zeroF reneD the referene mesure π(X) will only in)uene the dynmis of the proessD ut not the solutionF he energy funtion is divided in two prtsX n externl energy termD U ext θext (X, Y) spatio-temporal marked point process model pigure SFIX esults of frme di'erening for omputing the ojet evidene termF @A smge of ots in welourneD eustrli eirus h8Y @A prme di'erening of two onseutive frmesY @A prme di'erening results fter pplying wter mskY @dA prme di'erening results fter pplying wter msk nd performing morphologil erosion nd losing opertions to smooth the oundries of the foreground regionsF whih determines how good the on(gurtion (ts the input sequeneD nd n inE ternl energy termD U int θ int (X)D whih inorportes knowledge out the intertion etween ojets in single frme nd ross the entire th onsideredF he totl energy n e written s the sum of these two termsX

U θ (X, Y) = U ext θext (X, Y) + U int θ int (X). @SFQA
he prmeter vetors of the externl nd internl energy terms re θ ext nd θ int respetively nd θ = {θ ext , θ int }F he similrity to the ojet detetion pproh desried in the previous hpter is evidentF xote however tht in this seD Y represents 3h imge ue ontining oth sptil nd temporl informtionD s opposed to 2h imge in ghpter RF purthermoreD X is omposed of on(gurtions of ojets whih exist in eh time frme tF pinllyD the internl energy U int θ int (X) must model not only the intertions of ojets within single time frmeD ut lso the intertions of ojets ross timeF sn the following setionsD the two prts of the energy re desried in detilF

U int dyn (u) = d(•, u, •) -dyn 0 if ∃ (v ∈ x t-1 nd w ∈ x t+1 ) sFtF d(v, u, w) ≤ dyn 0 0
otherwise @SFSA where dyn 0 is threshold tht desries how muh ojets n devite from the motion model ut still e wrdedF xote however tht this motion model does not restrit the veloity of the ojets in terms of mgnitudeF por instneD this energy term will rewrd eqully oth on(gurtions depited in pigure SFPF sf the mximum veloity of ojets is known prioriD it n e inorported into the model y imposing mximum distne etween the ojets v nd uD respetively u nd wF he energy term tht wrds on(gurtions whih follow the dynmi model is the sum over ll ojets in the on(gurtionX

U int dyn (X) = γ dyn u∈X U int dyn (u). @SFTA
his term fvors the retion of ojets where the dt evidene is redued ut the dynmi model motivtes the existene of n ojetF 5.1.1.2 Label persistence sn order to distinguish etween distint trjetories lel is dded to the mrk of eh ojetF his lel n e viewed s trjetory identi(erF hi'erent lels men di'erent trjetoriesF husD the numer of lels hs to e kept losely relted to the numer of trjetories in the dt setF sdellyD the lrge numer of ojets u sttered ross the imge sequene should e ssigned to rther smll numer of spatio-temporal marked point process model pigure SFPX wo exmples of trks over three frmes tht will e identilly reE wrded y the dynmi energy termD s this term fvors ojets tht follow onE stnt veloity modelD regrdless of the tul veloity of the ojetF he green rrow shows the diretion nd the mgnitude of the veloityF veftX he ojet depited hs slow veloity over the three frmesY ightX he ojet hs very high veloity over the three frmesF lelsF sn this regrdD the set of lels present in on(gurtion X is onstruted y labels(X) = u∈X l(u)D where l(u) is the lel of ojet uF gon(gurtions where the numer of distint lels is smll re fvored in the following wyX

U int label (X) = -γ label 1 |labels(X)| @SFUA
where |labels(X)| represents the rdinlity of the setF

Mutual exclusion

rndling ojet ollision or overlpping in given frme is ruil spet when deteting nd trking ojetsF sn this modelD n in(nite penlty is ttriuted to ny on(gurtion tht ontins ojets tht overlp more thn given extent sF husD the proility of seleting suh on(gurtion is zeroF he sme nonEoverlpping onstrint s in etion RFIFPFI is usedF es reminderD denoting y A(u i , u j ) =

Area(u i ∩u j ) min(Area(u i ),Area(u j )) the re of intersetion etween the ojets u i nd u j D with u i , u j ∈ x t D the nonEoverlpping energy term n e de(ned sX

U int overlap (X) = t∈{0,••• ,T } 1≤i =j≤n(xt) t s (u i , u j ) @SFVA 5.1. Model 95 withX t s (u i , u j ) = 0 if A(u i , u j ) < s +∞ otherwise @SFWA
where s ∈ [0, 1] orresponds to the mount of overlpping llowed y the model nd n(x t ) is the numer of ojets in the on(gurtion x t t time tF his energy term is used to restrit the numer of ojets tht over the sme lotion in given frmeF

E(u|Y) = - 1 |u| p∈u 1{ν(p) = f oreground|Y} @SFIHA
where |u| mrks the rdinlity of ojet u @eFgF the numer of pixels tht elong to uA nd 1{•} mrks the inditor funtion @1{true} = 1D 1{false} = 0AF he ojet evidene E(u|Y) is used to fvor the detetion of smller ojetsF spatio-temporal marked point process model pigure SFQX @A e typil exmple of otY @A sn redX the exterior order F ρ (u) nd in greenX the interior order I 1 (u) used to ompute the ontrst distne mesure for ojet detetion in ghpter RY @A sn redX the exterior order F ρ (u) used to ompute the ontrst distne to the interior of the ellipse for trking purposes in this hpterF 5.1.2.1.2 Contrast distance measure he im of this term is to further re(ne the detetion nd extrt informtion suh s the orienttion nd the size of the ojetsF he ojets of interest pper s right strutures on drk kgroundF reneD ontrst distne mesure is omputed etween the interior of the ellipse nd its orderF he ontrst distne mesure hs een previously de(ned in hpter RF roweverD due to the existene of the evidene termD the impt of the ontrst distne is gretly reduedD s ompred to the previously desried modelF husD only the ontrst distne etween the interior of n ojet uD nd its ρEorder F ρ (u) is omputed s followsX

U ext (u, Y) = Q d B (u, F ρ (u)) d 0 (Y) , @SFIIA
where gin d B (•, •) is the ontrst distne mesure de(ned in eqF RFT nd Q(•) is the qulity funtion de(ned in eqF RFUF e reminder of the grphil representtion of the order is shown in pigure SFQF he two terms omputed in eqF SFIH nd eqF SFII re further omined into lol externl energy for n ojet uX

U ext local (u|Y) = γ ev E(u|Y) + γ cnt Q d B (u, F ρ (u)) d 0 (Y) . @SFIPA
pinllyD the externl energy term for the on(gurtion X isX

U ext θext (X, Y) = u∈X U ext local (u|Y). @SFIQA
he prmeter vetor of the externl termD θ ext = {γ ev , γ cnt }D onsists of the weight γ ev of the evidene term nd the weight γ cnt of the qulity of the ontrst distneF 5.1. Model 97

Statistical model

his method relies on fyesin deision theory nd onsists in sttistilly testing two hypotheses nd hoose the most likely oneF sn our settingD the im is to deterE mine whether n ojet is present or sentF pirstD we ompute the di'erene etween two onseutive frmesD F D t F his will result is gry level imge in whih the intensity of pixel represents the mount of hnge present t tht lotion etween the frmesF he lrger the mount of hnge @respF the higher the intensity of pixelAD the more likely it is for moving trget to e present in tht lotionF o determine thisD we use sliding window uD whih is trnslted over every pixel pF e hve the following two hypothesesX

• H 0 X the ellipse overs only the kground without ny trget eing present nd denote the re with AY

• H 1 X the ellipse is pled in the enter of trgetF he re orresponding to the interior of trget is denoted I u D while the re orresponding to the exterior of trget is denoted F u F e re interested in the following proilitiesX

P (H 0 |F D t )X he proility of H 0 D knowing the sene F D t Y P (H 1 |F D t )
X he proility of H 1 D knowing the sene F D t F @SFIRA e de(ne the proilities rtio in the following wyX

R = P (H 1 |F D t ) P (H 0 |F D t ) = P (F D t |H 1 ) P (F D t |H 0 ) P (H 1 ) P (H 0 ) , @SFISA
where P (H 0 ) is the proility tht the window u overs only the kgroundD independent of the oservtion nd P (H 1 ) is the proility of trget eing loted t the enter of the windowD independent of the oservtionF e de(ne the likelihood rtio

Λ = P (F D t |H 1 ) P (F D t |H 0 ) . @SFITA
he deision of whether trget is or not present t given lotion is tken s followsX • he intensity of eh pixel in F D t is independent nd uniformly distriutedY

Λ > P (H 0 ) P (H 1 ) (⇔ R > 1) e trget is presentY Λ < P (H 0 ) P (H 1 ) (⇔ R < 1)
• ih pixel within trget follows qussin distriution of unknown prmE eters @men nd vrineAY

• ih pixel within the kground follows qussin distriution of unknown prmeters @men nd vrineAY

• he vrine within the trget nd the kground is equlF yn the one hndD it is essentil to relize tht these onstrints re not very restriE tive when smll sliding window is usedD even in the presene of inhomogeneous kgroundF yn the other hndD it is ler tht this model is not well dpted when the kground presents visile textureF e will see lter why the lst onstrint is very usefulF fy using this modelD we n write the proility of pixel p from n re S sX

P (p) = 1 2πσ 2 S exp - (p -m S ) 2 2σ 2 S , @SFPHA
where σ 2 S is the vrine of n re S nd m S is the expettion of the re SF es the pixels re independentD we n write the proility of the re S sX

P (S) = i∈S 1 2πσ 2 S exp - (p i -m S ) 2 2σ 2 S @SFPIA
nd the logrithm sX

L(S) = - 1 2 p i ∈S log(2πσ 2 S ) + (p i -m S ) 2 σ 2 S .
@SFPPA e n now ompute P (F D t |H 0 ) nd P (F D t |H 1 ) using eqF SFPI nd eqF SFPPX

P (F D t |H 0 ) = P (A) ⇔ log(P (F D t |H 0 )) = L(A) @SFPQA P (F D t |H 1 ) = P (I u ) × P (F u ) ⇔ log(P (F D t |H 1 )) = L(I u ) + L(F u ). @SFPRA
reneD the logrithm of the likelihood rtio n e written sX

log Λ = L(I u ) + L(F u ) -L(A). @SFPSA
es the quntities m S nd σ S re unknown @here S stnds for the res AD I u nd F u AD lssil solution to solve this prolem is to reple them y their mximum likelihood estimtes mS nd σS respetivelyF efter few simpli(tions we otin the following expression for eqF SFPSX he hoie of the threshold depends entirely on the pplitionF e hve omputed threshold τ for eh dt set @the di'erent dt sets will e presented in detil in etion SFRAF e hve omputed the yg @eeiver yperting ghrteristiA urves for windows of size 9 × 9D 11 × 11 nd 15 × 15 in order to hoose the pproprite oneF he yg urves for the syntheti iologil dt set re displyed in pigure SFRF e oserve tht the est results re otined for window size of 9 × 9 pixels @red urveAF he sme window size is used for stellite dtF husD we n now set the threshold sed on the desired level of flse lrmsF sn our experimentsD we hve hosen τ s the threshold whih mximizes the proility of detetion @hA with the smllest proility of flse lrms @peAF sn pigure SFS we show detetion exmple on syntheti iologil imgeF imilr results re otined for stellite imgesF rere gin window size of 9 × 9 pixels gives the est resultsF e show detetion exmple on the oulon dt set in pigure SFTF

log Λ = - 1 2 N Iu log(σ 2 Iu ) + N Fu log(σ 2 Fu ) -N A log(σ 2 A ) @SFPTA whereX E N Iu D N Fu D N A re

External energy formulation

he externl energy term for n ojet u = (x u , y u , t u , a u , b u , ω u , l u ) is written s followsX

U ext stat (u) = -1 if M tu GLRT (x u , y u ) > τ, +1 if M tu GLRT (x u , y u ) < τ.
@SFPVA he externl energy term for on(gurtion X ssoited to the preEdetetion result isX

U ext stat (X) = γ stat u∈X U ext stat (u) @SFPWA
he prmeter vetor for the externl energy for this model is θ ext = {γ stat }F 102 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model

Total energy term

he totl energy term n e written s sum of ll the energy terms de(ned in setion SFIFP nd setion SFIFIX

• Quality modelX

U θ (X, Y) = γ dyn U int dyn (X) + γ label U int label (X) + γ o U int overlap (X)+ γ ev E(u|Y) + γ cnt u∈X Q d B (u,F ρ (u)) d 0 (Y) . @SFQHA • Statistical modelX U θ (X, Y) = γ dyn U int dyn (X) + γ label U int label (X) + γ o U int overlap (X)+ γ stat U ext stat (X|Y).
@SFQIA en intuition of how eh energy term in)uenes the output result is presented in pigure SFUF e totl of (ve @respF fourA weight prmeters γ • re needed to lne the individul terms in the (nl energy of the ulity model @respF ttistil modelAF sn ghpter R it ws rgued tht the stility of the model depends on the numer of prmeters tht hve to e estimted when iwElike lgorithms re used for the estimtionF he proposed energy ontins high numer of prmeters whih nnot e set empirillyF reneD lterntive prmeter lerning tehniques re disussed in the following setion in order to improve the stility of the model nd redue the omputtionl urdenF

Parameter estimation

rmeter lerning using iwElike estimtion lgorithms tends to eome unstle s the numer of prmeters inresesF woreoverD the omputtionl omplexity of suh methods inreses exponentilly with the numer of prmeters whih results in very high omputtionl lodsF husD lterntive lerning tehniques hve to e onsideredF e key property of the energy desried in eqF gFR nd eqF gFS is its linerity in the weight prmetersF his property is ruil in determining suitle prmeter lerning lgorithmF he linerity is exploited in this setion y proposing prmE eter lerning proedure sed on liner progrmmingF efter rief introdution to liner progrmmingD the estimtion of the weight pE rmeters is disussedF 5.2.1 Linear programming everl optimiztion prolems fll into the liner progrmming lssD mong whih re minimum spnning trees or shortest pthsF fut proly the most widely known prolem is the mximum )ow G minimum ut prolem in grph theoryF viner progrmming ws formlized nd pplied ehind the sron gurtin y untorovih to prolems in eonomisD while in the western soietyD uoopmns IWSI pplied it to shipping prolemsF viner progrmming prolems onsist of liner ojetive funtion whih hs to e minimized @or mximizedA sujet to numer of onstrintsF he onstrints re liner inequlities of the vriles used in the ojetive funtionF e liner progrmming prolem with n vriles nd m onstrints is in stndrd form when it is written sX

wximizeX n i=1 c i x i @SFQPA ujet toX n i=1 a ij x i ≤ b i , j = 1, • • • , m @SFQQA x i ≥ 0, i = 1, • • • , n @SFQRA
sf the interest lies in minimizing n ojetive funtionD the prolem n e rewritten in stndrd form y negting the oe0ients c i F he liner progrm n lso e written in vetor formX wximizeX a T x @SFQSA ujet toX A T x ≤ b, x ≥ 0. @SFQTA eny vetor x tht stis(es the onstrints of the liner progrmming prolem is lled fesile solutionF e liner progrmming prolem n e eitherX

• InfeasibleF sf there is no vetor x tht stis(es the onstrints of the liner progrmming prolemD the prolem is sid to e infesileY

• UnboundedF sf the liner progrmming prolem is not su0iently onE strined @for ny given fesile solutionD nother fesile solution tht furE ther improves the ojetive funtion n e foundAD the prolem is sid to e unoundedY

• Having an optimal solutionF he liner progrmming prolem hs n unique mximum @or minimumAF his does not men howeverD tht the vlues of the vriles tht yield the optiml solution re uniqueF he (rst lgorithm to solve liner progrmming prolems ws developed y hntzig in 1947 @hntzig IWRVD IWTQA nd is known s the implex lgorithmF he simplex method hs two stepsX (rstD fesile solution to the liner progrmming prolem is foundF yftenD trivil solution suh s x = 0 is fesile solutionF yne fesile solution is foundD the method itertively improves the vlue of the ojetive funtionF qrphillyD this n e seen s moving long the edges of fesile set from orner to ornerF wo issues hve to e kept in mind when solving liner progrmming prolems using the simplex methodX 104 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model

• InitializationF por mny liner progrmming prolemsD the trivil solution x = 0 is fesile solution to the prolemF xeverthelessD if this is not the seD fesile solution hs to e found to initilize the simplex methodF en uxiliry liner progrmming prolem n e formulted y sutrting ddiE tionl vriles from the onstrints of the originl prolem nd hnging the ojetive funtion to inorporte these new vrilesF he solution otined y solving the uxiliry prolem @with ll dditionl vriles eing zeroA is fesile solution for the originl prolemF sf no solution to the uxiliry proE lem is found where ll dditionl vriles re zeroD then the originl prolem is infesileF his proedure is lso known s the twoEphse simplex lgorithmY

• DegeneracyF sf orner on the edge of the fesile set is degenerteD the simplex method n get stuk is lol minimD simply euse ll the neighE ors of the orner re identil nd thusD the ojetive funtion n not e improvedF sn terms of omputtion timeD theoretil studies show tht the simplex method is exponentil in the numer of vrilesF xeverthelessD suh exponentil exmples rrely pper in prtieF he generl onsensus is tht the numer of itertions of the simplex method inreses linerly with the numer of onstrints nd only logrithmilly with the numer of vrilesD whih is why the simplex method is so widely usedF eentlyD liner progrmming pprohes hve experiened n inrese of populrE ity for multiple ojet trkingF ting et lF PHHU propose liner progrmming relxtion sheme for multiple ojet trking prolems with onvex interEojet intertion metri where the intrEojet stte ontinuity n e used s metriF ferlz et lF PHHW propose to use liner progrmming to solve the dt ssoition in trkingEyEdetetion frmeworkF hey suggest greedy serh pproh y reformulting the dt ssoition s onstrint )ow optimiztion prolem resultE ing in onvex prolem for whih the uthors use liner progrmming to solve itF pinllyD wvughlin et lF PHIS propose to enhne liner progrmming pprohes with motion modeling for multiple ojet trkingF

Weight estimation of individual energy terms as a linear programming problem

iqution gFR nd eqF gFS re liner in the weight prmetersF husD liner progrmming pproh n e envisioned for lerning these weightsF sn order to pose the prolem of weight estimtion s liner progrmming prolemD n ojetive funtion needs to e formulted nd the onstrints under whih to optimize it must e determinedF yn the one hndD the posterior density funtion is liner omintion of the weight prmeters for ny given on(gurtion XF yn the other hndD s shown in ghpter QD only the rtio π(X )/π(X) hs to e omputed in the wrkov hin trnsition without hving to know the ext vlue of neither π(X ) nor π(X)F trting from these propertiesD set of onstrints π(X )/π(X) ≥ 1 @or ≤ 1A n e estlishedD if one on(gurtion is known to e etter thn notherF hese onstrints n then e trnsformed into liner inequlities of the weight prmetersF pinllyD fter olleting su0iently mny liner inequlitiesD liner progrmming n e pplied to (nd fesile solution for the weight prmetersF yne wy to know whether one on(gurtion is etter thn nother is to strt from smll smple of ground truth informtionF qround truth informtion is ssumed to e ville nd the ground truth on(gurtion is denoted with X * F he ground truth on(gurtion X * ontins trks with orret lels nd lotionsF hi'erent perturtion kernels re pplied to degrde the ground truth on(gurtion to the on(gurtions X i F e generl overview on perturtion kernels hs een given in etion QFQ nd more detiled disussion of perturtion kernels for ojet trking will e given in etion SFQF gurrentlyD it is importnt to know tht the perturtions re degrding the ground truth on(gurtion to less optiml oneF es suhD for eh on(gurtion X i D the following onstrint holdsX

π(X * ) π(X i ) ≥ 1. @SFQUA
qiven one on(gurtionD the log funtion of the posterior f (C|X)

not.

= log(h(X|Y)) is liner funtion in terms of the weight prmetersF iqution SFQU provides one suh liner inequlityD iFeF f (C|X * ) -f (C|X i ) ≥ 0F sndeedD y olleting multiple onstrints the weight estimtion prolem n e posed s liner progrmming prolem s followsX

wximizeX a T C @SFQVA ujet toX A T C ≤ b, C ≥ 0, @SFQWA where C = [γ dyn , γ label , γ o , γ ev , γ cnt ]D a = [1, 1, 1, 1, 1] T for the ulity model @respF C = [γ dyn , γ label , γ o , γ stat ]D a = [1, 1, 1, 1]
T for the ttistil modelA nd eh row of the form A T C ≤ b enodes one onstrint from eqF SFQUF gre must e tken when dding inequlities to the set of onstrintsF es the ground truth informtion my ontin smll miguitiesD smll numer of on)iting onE strints my existF st is ruil to identify these on)iting onstrints in order to ensure tht the ojetive funtion is fesileF husD ny onstrint tht is in on)it with the urrent set of onstrints is ignoredF yne the set of onstrints hs een retedD the simplex lgorithm is used to (nd the est solution to the optimiztion prolemF xote tht ny vetor a ontining posiE tive numers is solution to the prolemF husD it is importnt to determine how mny inequlities re neessry to ensure n urte estimte of the prmetersF sn order to determine the neessry numer of onstrintsD we simulte density funtion with 5 prmeters @for the ulity modelA nd respetively 4 prmeters @for the ttistil modelAF por given numer of onstrintsD we independently generte multiple sets of onstrints nd evlute the verge normlized error s proposed y u nd wedioni PHHWF pigure SFV depits the orreltion etween the spatio-temporal marked point process model pigure SFVX he verge normlized error || Ĉ -C||/||C|| with respet to the numer of onstrintsF numer of onstrints nd the verge normlized error || Ĉ -C||/||C||F fsed on these resultsD we set the numer of onstrints to 6.000 in our experiments for the ulity model nd 5.000 onstrints for the ttistil model s the verge normlE ized error does not derese signi(ntly y using higher numer of onstrintsD s shown in pigure SFVF yne the prmeters of the model hve een trinedD the density funtion n e optimized to determine the est on(gurtion of ojets tht desrie given dt setF sn the following setionD we will desrie the optimiztion proedure used to simulte the sptioEtemporl models presented in this hpterF 5.3 Optimization he energies desried in eqF gFR nd eqF gFS re lerly not onvexF st is esy to onstrut exmples tht hve two virtully equl minimD seprted y wll of high energy vluesF he dependene used y the highEorder physil onstrints is the min reson tht drives the energy to e nonEonvexF he trget distriution is the posterior distriution of XD iFeF π(X) = f (X|Y)D deE (ned on union of suspes of di'erent dimensionsF he most widely known optimiztion method for nonEonvex energy funtions nd n unknown numer of ojets is the reversile jump wrkov ghin wonte grlo @twgwgA smE pler developed y qreen IWWSF twgwg uses mixture of perturtion kernels

Q(•, •) = m p m Q m (•, •)D m p m = 1 nd Q m (X, X )µ(dX ) = 1D
to rete tunE nels through the wlls of high energy @see etion QFQ for further detilsAF imulted nneling is used to (nd minimizer of the energy funtionF he density pigure SFWX @A ixmple input imge from the welourne dt set @ eirus h8AY @A ixtrted wter re using the lgorithm presented in RY @A snitil irth mp otined using simple threshold on the input imge nd restrited to the wter re using the wter mskY @dA isuliztion of n event oneF he one rehes oth forwrds nd kwrds in timeF he event one is used to inrese the proility of detetion in the volume it in)uenesF funtion in eqF gFI n e rewritten sX

f θ,i (X = X|Y) = 1 c T emp i (θ|Y) exp - U θ (X,Y) T emp i @SFRHA
where T emp i is temperture prmeter tht tends to zero when i tends to in(nityF sf T emp i dereses in logrithmi rteD then X i tends to glol optimizer of f θ,i F sn prtie howeverD logrithmi lw is not omputtionlly fesile nd heneD geometri lw is used insted to derese the tempertureF hereforeD proper design of the perturtion kernels is needed to ensure good explortion of the stte speF 5.3.1 RJMCMC in 2D + T he e0ieny of twgwg depends on the vriety of the perturtion kernelsF he stndrd perturtion kernels desried in ghpter Q re su0ient to ensure the onvergene of the twgwg smplerF xeverthelessD onvergene n e very slow when using only stndrd kernelsF husD the design of dpted perturtion kernels is ruil to ensure good onvergene propertiesF he following perturtion kernels re used for joint ojet detetion nd trkingX

• Birth and death according to a birth mapX es opposed to the lssil irth nd deth kernelD the use of irth mps inreses the e0ieny of the optimiztion 108 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model sheme y proposing irths nd deths in res where ojets re more likely to existF wo types of mps re reted in preEproessing stepX IF Birth mapsX sine ojets re supposed to hve higher rdiometri vlE ues thn the kgroundD simple threshold tehnique is used to identify prole lotions of ojets in every frme nd ttriute higher proE ilities to these lotions for the irth proposition kernelF en exmple mp for stellite imges is shown in pigure SFW @AY PF Water maskX sn the se of trking ots in stellite imges the wter re n e deteted s shown in pigure SFW @A nd the serh n e limited to suh resF terGvnd disrimintion ws disussed in etion RFQFQF his msk is only omputed for the welourne dt setF es reminderD the irth nd deth ording to irth mp kernel (rst hooses with proility p b nd p d = 1 -p b whether n ojet u should e dded to @irthA or deleted from @dethA the on(gurtionF sf irth is hosenD the kernel genertes new ojet u ording to the irth mp nd proposes X = X∪uF sf deth is hosenD the kernel selets one ojet u in X ording to the irth mp nd proposes X = X \ uY

X i = X is epted with proility α m = min(1, R m (X, X ))F ytherwise X i = X
oert nd gsellD PHHS @see etion QFQ for further detilsAF en importnt spet of multiple ojet trking is not only the onseutive detetion of the ojets within the imge sequeneD ut lso the orret leling of the ojets suh tht onsistent trjetories n e extrtedF sn our settingD the lel of the ojet determines the trjetory to whih it elongsF he perturtion kernels presented ove do not t on the lel of the ojets to mke them more onsistentF e hve nlyzed two di'erent pprohes to ojet relelingX IF Split/Merge perturbation movesF e split move onsiders n existing trE jetory nd proposes to split it in two di'erent trjetories t given split 5.3. Optimization 109 pointF e merge move proposes to unite two distint trjetories nd relE el the ojets suh tht they form singleD longer trkF e desrie the two perturtion movesD together with the ssoited qreen rtios in etion SFQFQFIY PF Deterministic labeling during the Birth moveF his leling pproh onsists in ssigning the lel of newly reted ojet sed on its neighE orhoodF hen new ojet u is retedD serh is performed to (nd the nerest ojet v in its neighorhood nd the lel of u is set to the lel of vF he omputtionl speedEups oupled with the good performne otined motivte the use of suh n pprohF purther detils re given in etion SFQFQFPF sn the following setionD we de(ne the perturtion kernels used for ojet trking nd give their eptne rtios in the prtiulr se of ellipsesF e then disuss the leling pprohes in etion SFQFQF

Proposition kernels used for object tracking

e propose to perform the optimiztion of the sptioEtemporl mrked point proess model presented in this hpter using wetropolisErstingsEqreen @or twgwgA smpler whose generl struture hs een introdued in ghpter Q @etion QFQAF es explined in etion QFQFID n interesting property of this smpler is the ft tht the proposition kernel Q n e deomposed into set of suEkernelsD eh orresponding to reversile perturtionF elthough single firth nd heth kernel is su0ient to gurntee the onvergene of the smpler @qreenD IWWSAD it is importnt to onstrut meningful perturtions in order to speed up the onvergene of the wrkov hinF sn this setionD we present di'erent suEkernelsX irth nd deth ording to irth mpD irth nd deth in the neighorhoodD lol trnsformtions nd ojet relelF he eptne rtios for ll perturtions re lso desried in this setionF he irth nd deth moves hnge the dimension of the on(gurtion y dding or deleting n ojetD while lol perturtions nd ojet relel hnge the prmeters of existing ojets without modifying the numer of ojets in the on(gurtionF gonsider the sptioEtemporl mrked point proess models proposed in this hpterF he ojet spe is he kernel proposes new on(gurtion X ∪ u nd the qreen rtio is given yX

W = K × M ⊂ R 2 × {1, • • • , T } × R 4 nd n ojet u is given y u = (x u , y u , t u , a u , b u , ω u , l u )D
R BM (X, X ∪ u) = h(X ∪ u) h(X) p d p b ν(W )b tu (x u , y u ) v∈X b tv (x v , y v )
. @SFRIA Death:

he kernel proposes new on(gurtion X \ u nd the qreen rtio is given yX

R DM (X, X \ u) = h(X \ u) h(X) p b p d v∈X (b tv (x v , y v )) ν(W )b tu (x u , y u ) . @SFRPA
• Birth and death in a neighborhoodF he irth nd deth kernel n e written s

Q BDN (X, •) = p b (X)Q BN (X, •) + p d (X)Q DN (X, •)D
where gin p b = p d = 0.5 represent the proility of irth @respF dethA nd Q BN (X, •) is the irth suEkernel whih dds new ojet to the on(guE rtionD while Q DN (X, •) is the deth kernel whih removes n existing ojet from the on(gurtionF e re interested in the following neighorhood reltion

∼ t X u ∼ t v ↔ ||u -v|| 2 ≤ d max with u ∈ x t , v ∈ x t±1 ||u|| 2 = x 2 + y 2
@SFRQA he neighorhood reltion ∼ t exists etween ojets in djent frmesF d max desries the mximum distne t whih n ojet v n e loted in n djent frme of ojet u nd still e in reltion to uF his kernel n e synthesized s followsX Birth:

IF ghoose n ojet u mong X = {x 1 , • • • , x T }D with x t = {u 1 , • • • , u n(xt) }D t ∈ {1, • • • , T } ording to the disrete lw η X b (•)Y PF qenerte v suh tht v ∈ ∂(u)
Σ → W @SFRRA ξ u : z → v. @SFRSA
sn our seD we n writeX

Σ = {(x, y) ∈ R 2 ||(x, y)|| 2 ≤ d max } × {t ± 1} × M @SFRTA nd z = (x, y, t, a, b, ω, l) ξ u (z) = (x u + x, y u + y, t u ± 1, a, b, ω, l). @SFRUA
he toin in this se is 1 nd if uniform genertor is used on ΣD then the qreen rtio n e written sX

R BN (X, X ∪ v) = h(X ∪ v) h(X) p d p b 1 2πd 2 max η X∪v d (v) u∈X η X b (u)1(||u -v|| 2 ≤ d max ) , @SFRVA where η X b (u) = 1 n(X) nd η X d (v) = 1 2 card{{u,w}∈N (X), v∈{u,w}} card{N (X)}
FD where N (X) represents the set of pirs {u, v} tht re in reltion ∼ t in XF he only missing ingredient is to uilt n uniform genertor within irleF he simplest solution is to generte ojets in squre of length 2d max until one ojet flls within the irleF Death:

IF ghoose n ojet u ∈ X suh tht ∂(u) ∩ (X \ u) = ∅ ording to the lw η X d (•)Y PF ropose X = X \ vD where v is hosen uniformly from ∂(u)F he qreen rtio in this se n e written sX R DN (X, X \ v) = h(X \ v) h(X) p b p d 2πd 2 max 1 u∈X η X\v b (u)1(||u -v|| 2 ≤ d max ) η X d (v) @SFRWA
• Local perturbationsF mll perturtions of n ojet re more e0ient thn deth immeditely followed y irth in terms of omputtion timeF his type of perturtions re symmetri in order to gurntee the reversiility of the hinF vet T = {T a : a ∈ A} e fmily of symmetri perturtions prmetrized y aF he kernel ssoited to this fmily onsists in rndomly hoosing n ojet u ∈ X nd proposing perturtion y pplying T a to uF sf oth the ojet u nd the trnsformtion T a re uniformly hosenD then the qreen rtio n e written sX

R L (X, (X \ u) ∪ v) = h((X \ u) ∪ v) h(X) . @SFSHA
Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model e hve used three types of lol perturtionsX trnsltionD rottion nd sleF e translation is prmetrized y vetor dx,dy] orresponds to trnsltion of the enter (x u , y u ) of the ellipse u suh tht the new enter is still in

[d x , d y ]D with d x ∈ [-δ x , δ x ] nd d y ∈ [-δ y , δ y ]F e trnsition T [
K = [0, I hmax ] × [0, I wmax ]X T [dx,dy]            x u y u t u a u b u ω u l u            =            (x u + d x )[I hmax ] (y u + d y )[I wmax ] t u a u b u ω u l u            @SFSIA
he fmily of rotations is de(ned in [-δ ω , δ ω ] nd rottion T dω modi(es the orienttion ω of n ojet u s followsX

T dω            x u y u t u a u b u ω u l u            =            x u y u t u a u b u (ω u + d ω )[π] l u            . @SFSPA pinllyD scale is prmetrized y vetor [d a , d b ] with d a ∈ [-δ a , δ a ] nd d b ∈ [-δ b , δ b
]F e sle perturtion modi(es the length of the semiEmjor nd semiEminor xis of n ojet u s followsX

T [da,d b ]            x u y u t u a u b u ω u l u            =            x u y u t u (a min + (a u -a min + d a ))[a max -a min ] (b min + (b u -b min + d b ))[b max -b min ] ω u l u            . @SFSQA

Consistent labeling

sn the previous setion we mentioned two wys of ssigning onsistent lels to the ojets within on(gurtion X = x 1 ∪ • • • ∪ x T F he (rst option is to rete two dditionl perturtion kernels tht will either split given trjetory in twoD or merge two given trjetories in single oneF he seond option is to deterministiE lly ssign lels to ojets t irthF foth pprohes hve their dvntges nd drwks whih we will disuss in the following susetionsF 5.3. Optimization 113 5.3.3.1 Split/Merge perturbations e develop perturtion kernel tht ts only on the lels of the ojetsF por thisD we de(ne reltion ∼ t suh thtX

u ∼ t v ⇔u ∈ x t , v ∈ x t+1 , ∀t ∈ 1, • • • , T -1 nd ||u, v|| 2 < ||u, w|| 2 , ∀w ∈ x t+1 , ||u|| 2 = x 2 + y 2 .
@SFSRA e de(ne the neighorhood ∂(u) of n ojet u in x t s followsX

∂(u) = {v ∈ x t+1 sFtF u ∼ t v}. @SFSSA e de(ne trjetory τ D τ = n(τ ) i=1 {u i }D suh tht u i ∼ t u i+1 nd label(u i ) = label(u i+1 ), ∀i = 1, n(τ ) -1D
where n(τ ) is the length of the trjetory τ F e de(ne mximl trjetory to e trjetory suh tht no ojet with the sme lel is in reltion ∼ t with the strting ojet u s t time t τ s or the ending ojet u e t time t τ e of the trjetoryX

τ = τ m ⇔ ∀v ∈ x t τ s -1 , v t u s nd ∀w ∈ x t τ e +1 , w t u e .
@SFSTA e n now de(ne reltionship ∼ etween two mximl trjetories τ m,1 nd τ m,2 if the lst ojet u

τ m,1 e is in reltion ∼ t with the (rst ojet u τ m,2 s X τ m,1 ∼ τ m,2 ⇔ u τ m,1 e ∼ t u τ m,2 s . @SFSUA vet Υ = {τ m,1 , • • • , τ m,k } e
T d l            x u y u t u a u b u ω u l u            =            x u y u t u a u b u ω u d l            .
@SFSVA e n now de(ne the splitGmerge perturtion kernel

Q SM (X, •) = p S (Υ)Q S (X, •) + p M (Υ)Q M (X, •)D s followsX • Split @Q S (X, •)AX IF elet mximl trjetory τ m Y PF elet split point u τ ording to proility lw η τ (u i )Y QF por ll ojets u i ∈ τ tht exist fter the split point u τ D pply trnsE formtion T d l Y RF ropose (X \ {u i ∈ {u τ , • • • , u τ e }}) ∪ {T d l (u i ∈ {u τ , • • • , u τ e })}Y 114
Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model

• Merge @Q M (X, •)AX IF elet two mximl trjetories τ m,1 nd τ m,2 suh tht τ m,1 ∼ τ m,2 Y PF por ll ojets u i ∈ τ m,2 D pply trnsformtion T d lτ m,1 Y QF ropose (X \ {u ∈ τ m,2 }) ∪ {T d lτ m,1 (u ∈ τ m,2 )}
e denote y n(Υ)D the numer of trjetories in ΥD y n ∼ (Υ)D the numer of seond order liques of trjetories in Υ with respet to the ∼Ereltion nd y n(τ ) the length of trjetory τ @more preiselyD the numer of ojets u ∈ X tht form the trjetory τ AF he proility of proposing split or merge n e omputed sX

p S (Υ) = n(Υ) n ∼ (Υ) + n(Υ) @SFSWA nd p M (Υ) = n ∼ (Υ) n ∼ (Υ) + n(Υ)
. @SFTHA he ssoited perturtion kernels reX

• sn se of splitX Q S (X, A) = 1 n(Υ) τ m,i 1 n(τ m,i ) u∈τ m,i 1 A ((X \ {u i ∈ {u τ , • • • , u τ e }}) ∪ {T d l (u i ∈ {u τ , • • • , u τ e })}) @SFTIA • sn se of mergeX Q M (X, A) = 1 n ∼ (Υ) τ m,i ∼τ m,j u∈τ m,2 1 A ((X \ {u ∈ τ m,2 }) ∪ {T d lτ m,1 (u ∈ τ m,2 )}).
@SFTPA he eptne rtio for the splitGmerge perturtion kernel depends on whether split or merge perturtion is proposedX

• sn se of splitD the qreen rtio is given yX

R(X, X ) = p M (X ) p S (X) h(X ) h(X) n ∼,X (Υ) (n X (Υ) -1)n(τ ) @SFTQA where X = (X \ {u i ∈ {u τ s , • • • , u τ e }}) ∪ {T d l (u i ∈ {u τ s , • • • , u τ e })}F
• sn se of mergeD the qreen rtio is given yX

R(X, X ) = p S (X ) p M (X) h(X ) h(X) n X (Υ)(n(τ m,1 ) + n(τ m,2 )) n ∼,X (Υ) + 1 @SFTRA where X = (X \ {u ∈ τ m,2 }) ∪ {T d lτ m,1 (u ∈ τ m,2 )}F
pigure SFIHX he stte of the wrkov hin depends on the order in whih the ojets re dded to the on(gurtionF gse IX X = ((X ∪ u) ∪ v) ∪ w @(rst uD then v nd then w ws retedAF gse PX X = ((X ∪ u) ∪ w) ∪ v @(rst uD then w nd then v ws retedAF 5.3.3.2 Deterministic label assignment e di'erent pproh of ssigning lels to the ojets in order to uild onsistent trjetories is y setting them in deterministi wyF reneD insted of rndomly ssigning lel to the ojet when it is retedD we determine the lel of the ojet t irthD sed on its neighorhoodF he lels re ssigned sed on the motion modelF qiven ojet u entered t lotion pos(u) = (c h (u), c w (u))D the ojets in the djent frmes re identi(ed tht re likely to stisfy the motion modelF he distne etween u nd these oE jets is omputed nd ompred to thresholdF sf the distne is smller thn the thresholdD the lel of ojet u is set to the lel of the ojet in the previous frmeF ytherwiseD new rndom lel from [0, L] \ labels(x t ) is ssigned to uF gon(gurE tions X tht ontin two or more ojets with the sme lel t ny time instne t re not permittedD mening tht n in(nite energy is ssigned to suh on(gurE tionsF he drwk of this pproh is tht the lels depend on the order of explortionF por instneD onsider three neighoring ojets uD vD w ording to some neighE orhood reltion

∼ t D suh tht u ∈ x t D v ∈ x t+1 D w ∈ x t+2 nd u ∼ t v nd v ∼ t wF
vet us nlyze the lels of the ojetsD depending on the order in whih they re retedX IF X = ((X ∪ u) ∪ v) ∪ w @(rst uD then v nd then w were retedAX sf we ssume tht u is not in reltion to ny ojet efore the retion of vD then the lel of u is set rndomlyD label(u) = lF hen ojet v is retedD it is in reltion with u nd thusD label(v) = label(u) = lF pinllyD when ojet w is retedD it is in reltion to ojet v nd heneD label(w) = label(v) = label(u) = lF sn the endD we otin on(gurtion with ll three ojets shring the sme lel lY PF X = ((X ∪ u) ∪ w) ∪ v @(rst uD then w nd then v were retedAX eginD if we ssume tht u is not in reltion to ny ojet t its irthD then the lel of u 116 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model is set rndomlyD label(u) = l 1 F xowD ojet w is dded to the on(gurtionF xote tht w ∈ x t+2 while u ∈ x t nd heneD u nd w re not neighorsF eordinglyD the lel of w is set rndomlyD label(w) = l 2 F pinllyD v is dded nd sine it is in reltion to uD the lel of v is label(v) = label(u) = l 1 F sn this se howeverD t the end we otin on(gurtion where the lels of the three ojets re not the sme nymoreF his simple exmpleD depited in pigure SFIHD shows tht the stte of the wrkov hin depends on the order in whih the ojets re dded to the on(gurtionF xeverthelessD s we will show in etion SFRD this pproh o'ers high trking ury while reduing the omputtionl urden ompred to the splitGmerge pprohF pinllyD the kernels presented in etion SFQFP together with the leling pE prohes hve een designed to improve the onvergene speed of the smplerF he smpler hs een emedded into simulted nneling sheme for etter perforE mneF xeverthelessD this lso led to n inresed omputtionl ostF sn order to tkle this prolemD we hve proposed two novel pprohesX

• Integrating Kalman like moves in RJMCMCF equentil (lters hve proven to provide reltively fst nd relile trking performnes in prE tiulr for single trget trkingF he properties of sequentil (lters n e e0iently exploited within the twgwg smpling shemeF he (lter is used to generte more meningful perturtion proposls whih re then evluted using n pproprite qreen eptne rtioF fetter perturtion proposls inrese the eptne proility of the overll twgwg smpling sheme whih in turn leds to fster onvergeneF

• Parallel implementation of RJMCMCF e prllel implementtion similr to the one presented in ghpter R is proposedF peil ttention needs to e given to ensure tht the prllel perturtions re independent in oth the sptil dimensions s well s in timeF he two tehniques re further investigted in this setionF pirstD the integrtion of sequentil (ltersElike moves within the twgwg smpling sheme is disussedF e dedited firth nd heth kernel using ulmnElike moves is introduedF he ulmn (lter is hosen for its omputtionl e0ieny nd ese of implementtionF roweverD prtile (lters n lso e used instedF xextD prllel implementtion of the twgwg smpler is proposedF he implementtion onsiders the perturtion kernels previously introdued in this setion nd does not extend to the hyrid irth nd deth kernel with sequentil (ltersElike movesF 5.3.4 Integrating Kalman like moves in RJMCMC es opposed to the lssil irth nd deth kernelD prolemEspei( irth nd deth kernel is proposed tht uses ulmn (lter within the irth stepF his llows to rete trklets @iFeF ellipses with the sme lel in onseutive frmesA in single stepF he ulmn (lter dtes k to 1960D when ulmn IWTH desried reursive solution to the disreteEdt liner (ltering prolemF his (lter eme very populr nd multiple vritions nd extensions of it hve een designed to djust the (lter to diverse prolems @frEhlom nd portmnnD IWVVD rugD PHIPAF 5.3.4.1 Kalman lter design he ulmn (lter @upA is pplied to estimte the stte of n ojetD where the stte is ssumed to e linerly qussin distriuted in timeF he ontinuity of the motion serves s strong predition riterion in ojet trkingF o generte new trkletsD the system is modeled s liner qussinD with the stte prmeters of the ulmn (lter given y the ellipse lotionD its veloityD its size nd its orienttionF por single ojetD the disreteEtime dynmi eqution is given yX

KX t+1 = F • KX t @SFTSA
where the stte vetor is given y KX = [c w , c h , ċ w , ċh , a, b, ω]D where c w nd c h re the predited oordintes of the ellipseD ċ w nd ċh re the veloities in the respetive diretionD a nd b re the semiEmjor nd semoEminor xis of the ellipse nd ω is the orienttionY nd

F =           
1 0 dt 0 0 0 0 0 1 0 dt 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1

           , @SFTTA
is the priori known trnsition mtrixD where dt is set to dt = 1 in the experimentsF he mesurement vetor M is otined using simple threshold pplied to the frme di'erening output to identify moving ojets in every frmeF et eh time frmeD the foreground los re identi(ed nd their enter lotionD widthD height nd orienttion re otinedF e mesurement vetor M t is onstrutedD tht n e injeted into the mesurement model of the upF eordinglyX

M t = H • KX t + q t @SFTUA
with q t ∼ N (0, R t ) eing white qussin noise with ovrine mtrix R t nd H is the mesurement funtion suh thtX

H =       
1 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1

      
. @SFTVA spatio-temporal marked point process model ine known model is ssumed for the dynmis of n ojetD the up n e used to predit the position of the ojet in the next frmeF he up stte predition KX t+1 nd the stte ovrine predition Pt+1 re de(ned yX

KX t+1 = F • KX t @SFTWA Pt+1 = F • Pt • F T @SFUHA
where KX t nd Pt re respetively the estimted stte vetor nd error ovrine mtrix t time tF henD the up updte step is s followsX

K t+1 = Pt+1 • H T H • Pt+1 • H T + R t+1 -1 @SFUIA KX t+1 = KX t+1 + K t+1 M t+1 -H t+1 • KX t+1 @SFUPA Pt+1 = (I -K t+1 • H) • Pt+1 . @SFUQA
he up strts with the initil onditions given y K 0 nd P0 F K t is lled ulmn gin nd de(nes the updting weight etween the new mesurements nd the preE dition from the dynmi modelF 5.3.4.2 Birth and death using the Kalman lter he Birth and Death using a Kalman lter kernel (rst hooses with proility p b nd p d = 1 -p b whether n ojet u should e dded to @irthA or deleted from @dethA the on(gurtionF sf deth is hosenD the kernel selets one ojet u in X nd proposes X = X \ uF roweverD if irth is hosenD the kernel genertes new ojet u nd proposes X = X ∪ uF sf the irth is eptedD ulmn (lter is initilized to the lotionD size nd orienttion of uF he stte t time t is updted using the mesurements M t nd then predition step is exeutedF he kernel genertes new ojet v sed on the stte vetor KX t+1 of the up nd proposes X = X ∪ vF he stte of the (lter is updted using M t+1 nd new predition is mdeF he proess is repeted until irth proposl is rejetedF xote tht the stndrd irth nd deth kernel is prtiulr se of the proposed perturtion kernelD when the proposl X = X ∪ v is rejetedF

Perturbation kernel and corresponding acceptance ratio

he firth nd deth using ulmn (lter n e divided into two stgesX

• rst stageX irth nd deth using irth mp perturtion is performedY

• second stageX depending on the outome of the (rst stgeD we hve the following sesX sf irth ws performed nd epted t the (rst stgeD then propose new irth sed on the ulmn (lter outputY ytherwiseD do nothingF 5.3. Optimization 119 he (rst stge of this kernel is identil to the irth nd deth using irth mp perturtion whih ws disussed in etion SFQFPF e n de(ne kernel for the seond stge of the perturtion s followsX

Q KF (X, •) = p b (X)Q BKF (X, •) + p d (X)Q DKF (X, •) @SFURA
where the irth kernel is de(ned using the predition of the ulmn (lterX

Q BKF (X ∪ u, A) = v∈W 1 A (X ∪ u ∪ v) N (v; KX v|u , P v|u )ν(dv) ν(W ) , @SFUSA
where N (v; KX, P) is the qussin proility density funtion with men KX nd ovrine mtrix P evluted t vY nd the deth kernel onsists in hoosing n ojet from the on(gurtion sed on the irth mp nd deleting itX

Q DKF (X, A) = v∈X 1 A (X \ v) b tv (x v , y v ) w∈X b tw (x w , y w ) . @SFUTA e n onsider the symmetri mesure φ(A × B) = C v∈W 1 A (X)1 B (X ∪ v)ν(dv)µ(dX) + C 1 A (X) v∈X 1 B (X \ v)µ(dX)F e hve
shown in ghpter Q tht this mesure is symmetri nd domintes π(dX)Q(X, dX )F reneD we n write the eptne rtios for the two ses s followsX BirthX

R(X ∪ u, X ∪ u ∪ v) = p d p b h(X ∪ u ∪ v) h(X ∪ u) b tv (x v , y v )N (v; KX v|u , P v|u ) w∈X b tw (x w , y w ) ; @SFUUA DeathX R(X ∪ u, X ∪ u \ v) = p b p d h(X ∪ u \ v) h(X ∪ u) w∈X b tw (x w , y w ) b tv (x v , y v )N (v; KX v|u , P v|u )
. @SFUVA enother wy to redue the omputtion time is to propose prllel implemenE ttion of the twgwg smplerF xowdysD onventionl omputers hve usully more thn two proessing units nd thusD prllel implementtions re eoming uiquitousF sn hpter R prllel implementtion of twgwg for ojet deteE tion in stti imges hs een desriedF he prllel implementtion is further extended to 2h C dimensions for trking purposesF

Ecient implementation of RJMCMC in 2D + T -multiple cores

he (rst mjor step towrds timeEe0ient pproh to simulte ws me with the development of the prllel smpler devised y erdié nd vfrge PHIPF he min ide in the 2D se is to divide the serh spe using qudtree KF he ells of the qudtree re divided into 4 independent sets lled miEsets nd denoted 120 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model S mic = S k D with k = 1, 4F et eh itertionD one miEset is seleted nd the optimizE tion step is performed in prllel within ll the ellsD cD ontined in itF he generl proposition kernel Q is formulted s mixture of uniform suEkernels Q c,γ D where γ ∈ Γ = {irth nd dethD trnsltionD rottionD sle}F he proility of eh proposition kernel is omputed s q c,t = P r(γ)

# cells in K F he omputtionl e0ieny of the smpler hs een proven for lrge numer of pplitionsF he smpler proposed y erdié nd vfrge PHIP mkes use of q omputing to perform the optimiztionF sn this workD multipleEore version of the smpler is implemented with shred Algorithm 9 rllel version of the twgwg smpler

IF snitilize X 0 = X 0 nd i = 0Y
PF gompute the wter msk @for the welourne dt set onlyAY QF gompute the dtEdriven spe prtitioning tree K nd trunte it sed on the wter msk @if it existsAY RF et itertion i with X i = XX

• por ll even frmesX

• ghoose miEset S mic ∈ K nd kernel type γ ∈ Γ ording to the proility c∈S mic q c,γ

• por eh ell c in S mic X ertur X in ell c to on(gurtion X ording to Q c,γ (X → •)Y etrieve the on(gurtion Z from the neighoring ellsY gompute the qreen rtioX

R = Q c,γ (X ∪ Z → X ∪ Z) Q c,γ (X ∪ Z → X ∪ Z) exp U (X ∪ Z) -U (X ∪ Z) T i ghoose X i+1 = X with proility min(1, R) nd X i+1 = X otherE wiseY • pdte T i+1 = αT i @in this work α = 0.95AY
• epet for uneven frmesF memory etween the oresF et eh itertionD for eh ell in hosen miEset S k D k = 1, 4D the on(gurtions within the neighoring ells re tken into onsidertionF xote tht this does not led to synhroniztion prolems when essing the shred memory sine proessors only red informtion in the neighoring ells nd do not modify itF he resoning ehind the use of multipleEore smpler ws desried in etion RFQFPF sn this setionD the prllel version of the twgwg smpler is extended to the 5.4. Results

121

2D + T frmeworkD the prllel smpler eing detiled in elgorithm WF he ide ehind this extension is sed on the ft tht independent perturtions in time n e performed for ny frme tD if the on(gurtions x t-1 nd x t+1 re kept onE stntF hese on(gurtions hve to remin onstnt euse the energy term tht inorportes the motion model into the energy for n ojet t time t is omputed using three onseutive frmesD s de(ned in eqF SFRF fsed on this oservtionD one itertion of the smpler n e divided in two prts s shown in elgorithm W @step 4AX (rstD perturtions re performed in prllel on ll even frmes in the th y keeping ll uneven frmes onstntY seondX the proess is repeted for uneven frmes y mintining the even frmes onstntF he two optimiztion pprohes desried in this setion re fundmentlly di'erE entD ut oth serve the sme purposeX fster optimiztion of the dedited energy funtion previously desriedF sn the following setionD results on vrious types of dt sets from two di'erent (eldsD remote sensing nd )uoresent mirosopyD re presentedF

Results

he proposed models were pplied to lrge vriety of dt setsD whih n e tegorized s followsX

• Synthetic benchmarks used by the biological processing communityF everl sequenes were generted with vrious levels of noise nd di'erent numer of ojetsF he ojets n enter or exist the region of interest t ny time nd lotionF hese dt sets hve een extensively used to nlyze the performne of the methodD given the existene of preise ground truth informtionY

• Real biological dataF yne sequene of rel iologil dtD provided y ourtesy of tF lmeroD sg sfieD w IRR gx snstitut gurie @fsset et lFD PHIRAD is used to show the ppliility of the proposed method for rel pplitions in the iologil ontextY

• Simulated satellite dataF hese dt sets re rel stellite imge sequenes whih were ltered to resemle the output of geosttionry stelliteF e lrge vriety of smples is explored rnging from stti mers to moving mersD high temporl frequenies to low temporl frequeniesD single ojet type to multiple ojet types tht hve to e deteted nd trked over the sequeneF he results for eh tegory of dt re illustrted using smll numer of repreE senttive exmplesF wore spei(llyD sine the enhmrk iologil dt n e generted with the ssoited ground truthD the limits of the proposed pproh re tested wFrFtF dense environments in sequenes with either very low temporl frequenies @1 -2rzA s well s very high temporl frequenies @up to 60rzAF spatio-temporal marked point process model

Tracking results on synthetic benchmarks used by the biological processing community

pigure SFIIX hetetion nd trking results otined on three syntheti iologil imge sequenes of 100 frmes eh @ sxseAF ih olor represents di'erent trkF yjets n pper nd dispper t ny lotion nd time instneF ih sequene hs di'erent level of qussin noise @leftX no noiseY middleX µ = 25D σ = 2.5Y rightX µ = 50D σ = 5.0AF

le SFIX untittive nlysis of the detetion nd trking results otined using the uiltEin wr trker within sy @de ghumont et lFD PHIPAD the ontinuous energy minimiztion lgorithm developed y wiln et lF PHIR nd the proposed method for the three syntheti iologil imge sequenesF he proposed lgorithm hs the highest similrity sores wFrFtF the ground truthD oth in terms of trks nd detetionsF he proposed lgorithm outperforms urrent stte of the rt methods y more thn 5%F he proposed pproh is (rst tested on three syntheti iologil imge seE quenesF he imge sequenes hve een generted with sy @de ghumont et lFD PHIPAD n online ville toolox for iologil imge sequenes developed y the untittive smge enlysis nit t the steur snstitute in risF he sfs gell rking ghllenge in 2013D led to the development of n sy plugEin lled 4sfs ghllenge4 whih n e used to generte syntheti sequenes with the ssoited ground truthF he sequenes onsist of 100 imgesD eh 512 × 512 pixelsD with pproximtely 35 ojets per frmeF he three sequenes exhiit di'erent levels of qussin noiseF rgets n pper nd dispper t ny lotion nd time instneF he extrted 5.4. Results

123

le SFPX hi0ult senrios with two or more rossing or yEpssing trjetories in syntheti iologil imge sequenesF pirst rowX qround truth trjetoriesF eE ond rowX rjetories otined using proposed method @ sxseAF he trjetories losely resemle the ground truth trjetoriesF vels re orretly preserved durE ing the rossings or yEpssingF hird rowX rjetories otined using wr @sy pluginAF he trjetories re highly frgmentedF he lels re swithed or ojets re lost during rossings or yEpssingF trks re depited in pigure SFII nd the quntittive trking results re displyed in le SFIF he ojets re onsidered to exhiit direted motionF 5.4.1.1 Tracking results of the models he proposed methods re ompred with the uiltEin wultiple rypothesis esting @wrA prtile trker tht omes s plugEin to the sy softwre he ghumont et lFD PHIPF he superiority of the proposed models n e esily oserved from the detetion nd trking similrity vlues in le SFIF he detetion nd trking similrity wFrFtF the ground truth is lmost doule s ompred to tht of the uiltEin trkerF he trking results of the wr plugEin hve lso een fed to the reently developed ontinuous energy minimiztion sheme proposed y wiln et lF PHIR for multiple trget trkingF his led to etter detetion nd trking resultsF roweverD the proposed pproh outperforms this result y more thn 5% in terms of detetion nd trk similrity with the ground truthF he joint detetion nd trking method extrts longD smooth trksF woreoverD it n orretly hndle ojet ppernes nd disppernesF sn le SFP severl zoomEins re shown on very di0ult senrios where two or more trjetories re rossing or pssing lose to eh otherF he proposed method orretly mintins the lels nd does not segment the trks s the wr trkerF spatio-temporal marked point process model ttistil model ulity model le SFQX ulity of the detetion using the ttistil model @topA nd the ulity model @ottomAF 5.4.1.2 Quality model vs. Statistical model he min di'erene etween the ulity model nd the ttistil model introdued in etion SFIFP lies in the ility to orretly identify the ojets nd determine the prmeters of the orresponding ellipses s losely s possile to the tul prmeters of the trgetsF yn the one hndD the ulity model hs two externl energy termsX one fvors the detetion of trget @the evidene termA nd the seond fvors strong lignment etween the ellipse nd the trget @the ontrst distne mesureAF reneD low externl energy will orrespond to on(gurtion tht hs the lrgest overlp with the trget on(gurtionF his model is very e0ient when the interest lies in oth trking the trgets nd segmenting them urtelyF roweverD it is di0ult to identify the proper prmeters of the model when predetermined proility of detetionGflse lrms is desiredF yn the other hndD the ttistil model hs single externl energy term tht links the motion present in onseutive frmes to the proility of deteting trgetF he model n e esily dpted to the needs of the user in terms of detetionGflse lrms proility y djusting the threshold τ used for the mtrix M GLRT @see etion SFIFPAF he drwk of this model wFrFtF the ulity model lies in the qulity of the detetionF heres this model only determines the preseneGsene of trgetD the ulity model produes etter segmenttion of the trgetF le SFQ shows the detetion results using the two modelsF e ler di'erene is oservle in terms of the qulity of the detetionF xeverthelessD the trking performne is not in)uenedD s shown in le SFIF 5.4.1.3 Split/Merge perturbation kernel vs. Deterministic labeling sn etion SFQ we hve developed twgwg frmework for simulting the proE posed modelsF en importnt spet in trking is the ility to ssign onsistent lels to ojets throughout the imge sequeneF es suhD we hve proposed two 5.4. Results 125 heterministi lels plitGwerge kernel @A @A le SFRX @A ixmples when the deterministi leling outperforms the splitGmerge pprohF @A ixmples when the reverse is trueF wys of ssigning the lels of the ojetsX splitGmerge perturtion kernel whih ts on the trjetories existing in on(gurtion nd deterministi pproh of setting the lels ording to prede(ned ruleF he splitGmerge perturtion kernel is in tone with the generl frmework presented in this workF st provides method to modify the lels of the ojets of the simE ulted hinF heterministi leling on the other hnd depends on the order of the explortion s shown in etion SFQF xeverthelessD deterministi leling pE proh signi(ntly redues the omputtionl urden tht is linked to performing splitGmerge movesF ixperimentl results hve shown tht the performne of the two leling pE prohes is omprle in terms of qulityF sn le SFR @A we show exmples when the deterministi leling outperforms the splitGmerge pprohD while in E le SFR @A we show the reverseF he susequent results showed in this setion hve een otined using the ulity model nd deterministi leling pprohD unless otherwise sttedF

Tracking results on real biological data

he min ojetive howeverD is to pply the proposed lgorithm on rel dtF husD the lgorithm hs een pplied on rel iologil sequene of 300 imges of 350 y 1340 pixelsD y ourtesy of tF lmeroD sg sfieD w IRR gx snstitut gurie @fsset et lFD PHIRAF he sp imge sequene shows ellD with the orresponding vesiles tht trnsport sustnes inside itF yne frme onsists of two imges of the ell tken t the sme time instne ut t di'erent wvelengthsF he vesiles n pper or dispper t ny lotion nd time instne sine frme ontins only 2h representtion of the tul 3h seneF he min ojetive is to detet nd trk these vesiles in order to understnd how they move within the ellF he x for this dt set is very lowF purthermoreD the intensity of the rdiometri 126 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model pigure SFIPX hetetion nd trking results on rel iologil sp sequene of 300 imges @y ourtesy of tF lmeroD sg sfieD w IRR gx snstitut gurie @fsset et lFD PHIRAA @ sxseAF he imge sequene shows ellD with the orresponding vesiles tht trnsport sustnes inside itF he gol is to detet nd trk these vesilesF he visul ssessment of the results revels their very good qulityF le SFSX gomprison etween the results otined using the uiltEin wr trker within sy @de ghumont et lFD PHIPA nd the proposed method for the rel sp imge sequene @fsset et lFD PHIRAF xote howeverD tht the output of the wr trker should not e tken s ground truth informtionF he visul ssessment of the results revels tht the trks otined using the proposed lgorithm re more onsistent nd less frgmented thn those otined using the wr plugEinF vlue of the vesiles fdes in time due to the dt quisition proessF he detetion nd trking results re depited in pigure SFIPF he visul ssessment of the results is very goodF vongD smooth trks re identi(ed whih orrespond to the liner motion of the vesilesF his result is in greement with urrent reserh in ell iology wFrFtF the motion of vesiles within ellF he proposed pproh is gin ompred to the uiltEin prtile trker of the sy softwreF roweverD in the sene of ground truth informtionD only the detetion nd trking similrity of this model wFrFtF the wr trker is shown in le SFSF 5.4.3 Tracking results on simulated low temporal frequency satellite data he quisition rte of stellite imges hs experiened signi(nt inrese in the lst yersF hereforeD ojet trking using high resolution stellite imges n e regrded s new pplition in remote sensingD omplementry to ojet 5.4. Results

127

detetion nd lndEover lssi(tionF reneD the proposed pproh is tested on hllenging rel high resolution optil stellite imge sequenesF he frme rte of these sequenes is round 1 -5rzF

MetricsF gonduting n ojetive omprison etween di'erent trking lgorithms is hllenging tsk for vrious resonsF pirstD the importne of individul trking filures is pplition dependentF eondD lssifying trker outputs s orret or inorret my s well e very miguous nd usully requires dditionl prmeters @eFgF thresholdsA to ssess the orretness nd preision of the trkersF o evlute the multiEojet trking uryD we ompute three types of errorsX flse positives @pAD flse negtives @pxA nd identity swithes @shAF he three types of errors re weighted equllyF he numer of true positives @A is lso stted nd the totl numer of moving ojets @yA is providedF he totl numer of moving ojets @yA is the sum over ll frmes of the ojets tht hnge their position in two onseutive frmesF edditionllyD mostly trked @wA nd mostly lost @wvA sores re omputed on the entire numer of distint trjetories @A to mesure how mny ground truth trjetories re trked suessfully @trked for t lest 80 perentA or lost @trked for less thn 20 perentAF pinllyD the preision @ G @ C pAA nd rell @ G @ C pxAA of eh lgorithm is sttedF he results re ompred to two lssil trkersX ulmn (lter nd smoother @elh nd fishopD PHHIA nd ristogrmEfsed rker @hll nd riggsD PHHSAF 5.4.3.1 Melbourne sequences wo high resolution optil stellite imge sequenes re onsidered for evlutionF ih sequene onsists of 14 frmes tken t low temporl frequenyF he sensor is moving t reltively slow speedF he quisition ngle of two onseutive imges hnges signi(ntlyF pigure SFIR shows the displement etween two frmesF

Object appearanceF yjets exhiit strong vritions in pperne due to the hngE ing ngle t whih the imges were tkenF le SFT @topA depits the pperne vrition of one ot over the dt setF roweverD the proposed method orretly identi(es the trget even under these onditionsD s shown in le SFT @middleAF his is minly due to the use of ontrst distne mesureD without tking into ount dditionl informtion within the ellipseF his is the min reson why the histogrmEsed trker fils to identify the ot s the sme ojet ross timeD s shown in le SFT @ottomAF Quantitative assessmentF pigure SFIQ shows the results of our method on the two imge sequenes onsideredD while the quntittive results for eh imge sequene re presented in le SFUF he results of three trkers re shownX the full model inluding dynmi irth mps nd the wter msk used for optimiztion presented in this hpterD denoted @Ew C fwAD the ulmn (lter nd smoother @upA nd the histogrmEsed trker @rfAF por omprisonD the detetion results of the sptil mrked point proesses model developed in hpter R nd denoted @wA whih ws pplied independently on eh frme to extrt ots is lso listed in le SFUF 128 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model he (rst imge sequene hs 14 frmes of 1840 × 820 pixels eh nd ontins le SFTX wo hrd ot detetion nd trking sesF yjet 1 lmost lends with the kground y the end of the sequene nd is hrd to distinguish from noise @wvesAF he proposed method suessfully detets the ojetD due to the joint optimiztion over detetions nd trksD while ll other trkers fil to distinguish the ojet from the kground noiseF yjet 2 exhiits strong pperne hnges due to its inrese in speedF he til of this ojet is mistken for new ojet y ll trkers exept the proposed methodF totl numer of 8 moving ojets throughout the entire sequeneF he use of the ojet evidene term leds to etter estimtes of the lotions of the ojetsD while the ontrst distne mesure is used to otin urte vlues for the size nd orienttion of the ojetsF woreoverD the lels re preserved throughout the imge sequeneF he seond imge sequene hs 14 frmes of 830 × 730 pixels ehF he sequene ontins totl numer of 4 moving ojets s well s lrge numer of stti ojets of the sme typeF he evidene term plys deisive role in distinguishing dynmi ojets from stti onesF he proposed model yields higher trking performne ompred to the lssil trkersD due to the etter detetion resultsF he lels of the ojets re generlly preserved throughout the sequeneF he proposed method @Ew C fwA outperforms oth the ulmn (lter nd smoother @upA nd the histogrmEsed trker @rfAF he lower performne of the ulmn (lter is desried y the lower performne of the detetor used whih produes high numer of flse lrms due to kground noise @wvesAF he performne of the histogrmEsed trker is highly in)uened y the hnge in illumintion due to the di'erent ngles of quisition of the stellite imgesF he pperne of the ojets hnges throughout the sequene nd thusD the preision pigure SFIQX hetetion nd trking results on two sequenes of rel stellite imges tken t di'erent ngles @ sxseAF ih olor represents di'erent trkF veftX rking results on the (rst imge sequene up to frme 10F ightX rking results up to frme 13 of the seond imge sequeneF le SFUX untittive results for the two sequenes of rel stellite imgesF he proposed method hs the highest preision nd the seond highest rell sores for the (rst sequeneD s well s the highest preision nd rell sores for the seond sequeneF he proposed methods sueeds in trking more trgets in eh sequene thn the other methods @wA nd lso hs the est detetion rtes for oth sequenes @AF pigure SFIRX hisplement etween two onseutive frmesF he ojets exhiit lrge jumps from one frme to the nextF histortions re lso visile on the lnd reF Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model of the trker is 'etedF sn terms of pperneD the proposed trker however only relies on the ontrst etween the ojets nd their orderF hereforeD its performne is not 'eted y pperne hngesF he sptioEtemporl model is more urte thn perEfrme detetion using sptil model @wAD sine the temporl model expliitly uses temporl informtion to inrese detetion resultsF he inresed qulity of detetions leds in turn to etterEformed trjetoriesF 5.4.3.2 Barcelona sequence -low temporal frequency he results on one high resolution sequene tken over the port re of frelon is nlyzedF he sequene ontins 420 frmesD eh 1600 × 900 pixels in sizeF vrge geometri distortions pper due to the movement of the imging sensorF he ojet evidene presented in setion SFIFP is sed on frme di'ereningF uh tehnique n e e0iently used when the mer is stti or nerEsttiF he performne of kground modeling pprohes drops with the inrese in mE er motionF reneD suh n pproh is not suited in this seF he frme difE ferening pproh desried in SFIFP is repled y more sophistited motion detetion lgorithm developed t eirus h8 whih tkes the geometri distortions into ountF he output of this lgorithm is fed lso to the ulmn (lter nd the ristogrm sed trker used to evlute the resultsF eprt from the geometri distortions whih re depited in pigure SFIS @AD the sequene lso exhiits high level of noiseF he noise is minly due to the wter wves whih led to very high flse lrm rte of the motion detetion lgorithmF Object appearanceF sn this sequeneD the interest lies in deteting nd trking ll ojets tht exhiit onsistent motion pttern throughout the sequeneF yjets re hrterized y signi(nt ontrst with the kgroundF woving ojets inlude rsD usesD ots nd pedestrinsF roweverD no distintion is mde eE tween the di'erent ojet typesF e postEproessing step n e used to distinguish etween the di'erent ojet lsses sed on their size nd veloityF Qualitative evaluationF he motion detetion lgorithm used in the evidene term of the model hs high flse lrm rteF his n e seen in pigure SFIS @AF ih irle is detetion nd the olor of the irle represents the proility of the deE tetion eing true trgetF e yellow irle shows smll proilityD wheres red irle shows high proility of eing trgetF he size of the irle reltes to the size of the ojet detetedF he mjority of the flse lrms re produed y wves in the seF his is typil prolem when deling with turulent se within sequene of imgesF pigure SFIS @A shows the )ow of the ojets during the entire sequeneF his pigure is prtiulrly importntD sine it visully demonstrtes tht the performne of the proposed model is not 'eted y the high numer of flse lrms otined while preEdeteting the trgetsF pigure SFIS @dAD shows the trks otined in the urrent frme using the proposed methodF foth up nd rf re strongly 'eted y the detetion resultsD yielding lrge numer of spurious trks on wterD s n e seen in pigure SFIS @eA nd pigure SFIS @fAF elthough the lk of ground truth dt does not llow n inEdepthD quntittive 5.4. Results

@A

@A @A @dA @eA @fA pigure SFISX @A smge displement etween two imges 20 frmes prt in the frelon sequene t 3rz temporl resolutionY @A hetetion results of possiE le moving trgets otined using the motion detetor provided y eirus h8F he detetor n hndle geometri distortions nd mer movementsF he irles represent possile detetions with vrious proilities @yellow a lowD red a highAY @A he motion )ow in the entire sequeneF vighter res highlight lotions with intense motion throughout the sequeneF he proposed model minly identi(es the ojets whih re on lnd with few flse lrms round the ostF roweverD y tking into ount the temporl informtionD the high numer of flse lrms generted y the detetion preEproessing step is onsiderly reduedY @dA rgets deteted t frme 170 of the sequene using the proposed methodF he distint olE ors of the ojets represent di'erent trksY @eA rgets deteted t frme 170 using the extended ulmn (lter nd smootherF ih ojet is ssigned numer to repE resent di'erent trksY @fA rgets deteted t frme 170 using the histogrmEsed trkerF ell ojets hve the sme olorF veling is not shown on this imgeF Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model nlysisD the proposed method outperforms visully the up nd rfF he proE posed lgorithm trks oth rs nd pedestrinsF es stted in the eginning of this setionD no distintion is mde etween di'erent lsses of ojets is impliitly mde within the modelF xeverthelessD postEproessing step sed on the size nd speed of the ojets n e used to distinguish etween ojet lssesF 5.4.4 Tracking results on simulated high temporal frequency satellite data ideo qulity n lso e otined y high resolution optil stellite sensors for short period of timeF sn this setionD severl videos re nlyzed when the imging sensor is oth stti nd movingF he frme rte is round 30rz for eh sequeneF 5.4.4.1 Toulon sequences wo sequenes re quired y stti mer over the oulon hror reF ih sequene ontins 150 frmesF qiven the ft tht the mer is sttiD the ojet evidene is determined sed on frme di'ereningD s shown in setion SFIFPF he ojets of interest re moving otsF he ots hve di'erent olorsD shpes nd n e prtilly oluded or lended with the kgroundF Qualitative and quantitative evaluationF he detetion nd trking results re shown in le SFVF he proposed lgorithm onsistently trks the moving ots over the entire sequene onsideredF e single ojet is missed y the lgorithm in the seond sequeneF his ojet moves very slowly nd thusD the frme di'erening pproh does not identify ny movement where tht ot is lotedF his n e lso oserved in the displement imge shown in pigure SFIUF reneD the evidene term hs very low vlue for this prtiulr ojetF es it is never deteted s possile ndidte ojetD the motion model does not help in inresing the deteE tion proilityF hereforeD this ojet is onsistently missedF foth the up nd rf lso miss this ojet in sequene 2F purthermoreD these trkers hve di0ulties identifying the moving ojet in sequene 1 s onsisE tent trgetD s it is onstntly prtilly oluded y the surrounding ojetsF he similrity of the moving trget with its surroundings leds to n unrelile lolE iztion of the trget y the rfF his similrity lso 'ets the proposed method whih is why the resulted trjetory is rther unstedyF xeverthelessD the proposed lgorithm is less sensitive to it thn the rfF 5.4.4.2 Barcelona sequences -high temporal frequency hree videos ptured y moving mer over the frelon re re nlyzedF hese sequenes lso su'er from lrge geometri distortionsF roweverD given the high temporl frequeny of the videosD the geometri distortions re reltively smll within nrrow th of frmesF he displement etween two imges 20 frmes prt in pormul 1 sequene is depited in pigure SFIT @AF he displement of the kground res is lmost nonEexistentF husD the evidene term still relies on pigure SFIUX smge displement etween two imges 20 frmes prt in the oulon sequene t 30rz temporl resolutionF he only visile displement is in the motion of the otsF xote howeverD tht the displement of the undeteted ot is not visile even fter 20 frmesF le SFVX untittive results for the two sequenes of stellite sequenes of high temporl frequeny @30rzAF he proposed method hs exellent preision nd etter rell thn the ulmn (lter nd the ristogrm sed trkerF xote tht ll trkers miss one ot @imge on the rightAF ine the veloity of this ot is very smllD frme di'erening fils to identify the ojet s foregroundF husD the ojet is never deteted throughout the sequeneF pigure SFIVX hetetion nd trking results on two sequenes of simulted stellite imges of oulonF he proposed method onsistently misses one ot @imge on the rightA euse of its very smll veloityF he imge sequenes re y ourtesy of eirus hefense 8 peD prneF 134

Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model frme di'erening to detet possile ojetsF @A @A @A @dA pigure SFITX @A hisplement etween two imges 20 frmes prt in sequene with high temporl frequenyF he imge sequene is y ourtesy of eirus hefense 8 peD prneF @A hetetion nd trking results for the frme 94F @A r0 density fter 500 frmesF @dA rking results otined y up in frme 94F

Qualitative evaluationF he detetion nd trking results re very goodF yn the one hndD the proposed method is very roust to noise nd outperforms the up nd rf in this regrdF pigure SFIT @A shows the trking results using the proE posed methodD while pigure SFIT @dA shows the trking results using upF e lrge numer of spurious trks re wrongly deteted y the upF yn the other hndD generl tendeny of the proposed model to miss ojets tht do no exhiit onE sistent pttern is oservedF he utomti prmeter lerning tehnique proposed in this hpter is sed on ville ground truth informtionF es the prmeters re lernedD they re optimized to (t n lmost zero flse lrm senrioF husD the model is more likely to dismiss ojets tht re not good (tsD leding to etter preision ut t the expense of rell rtesF en lterntive solution to inrese the detetion proility is the use of the ttistil modelF his model n e esily modi(ed to provide the desired detetionGflse lrms rtesF

Qualitative evaluation using the Statistical model vs. the Quality modelF he min dvntge of the ttistil model over the ulity model is the ility to set desired proility of detetionGflse lrmsF he prmeter lerning tehnique reE mins the sme nd thusD the prmeters otined (t n lmost zero flse lrms pigure SFIWX hetetion nd trking results for the frme 94 of the pormul 1 seE queneF opX esults otined using the ulity modelF he model o'ers good detetion rte with zero flse lrmsF fottomX esults otined using the ttisE til modelF he model o'ers etter detetion rte t the ost of n inresed flse lrms rteF he imge sequenes re y ourtesy of eirus hefense 8 peD prneF 136

Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model pigure SFPHX hetetion nd trking results on sequene of simulted stellite imges of frelonF he tr0 density fter 500 n e oservedF opX he tr0 density otined using the ulity modelF fottomX he tr0 density otined using the ttistil model with lrger numer of deteted trgets t the ost of high flse lrm rteF he imge sequene is y ourtesy of eirus hefense 8 peD prneF 5.4. Results

137

pigure SFPIX hetetion nd trking results on sequene of simulted stellite imges of frelonF he tr0 density fter 500 n e oservedF opX he tr0 density otined using the ulity modelF fottomX he tr0 density otined using the ttistil model with lrger numer of deteted trgets t the ost of high flse lrm rteF he imge sequene is y ourtesy of eirus hefense 8 peD prneF

Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model senrioF xeverthelessD in this seD the hoie of the threshold τ of the mtrix M GLRT n e modi(ed fter prmeter lerning to ensure etter detetion proE ilityF pigure SFIW shows the results otined on the sme pormul 1 sequeneD with higher proility of flse lrmsF yjets previously undeteted using the ulity model re suessfully deteted using the ttistil modelF roweverD the inresed detetion proility omes t the ost of higher flse lrms rteF wost of these flse lrms pper in the prking lotD due to the re)etion of the sunlight whih produes onsiderle mount of motion @di'ereneA etween onseutive frmesF pigure SFPH nd pigure SFPI show the tr0 density fter 500 frmes on two imE ge sequenesF he trker ssigns onsistent lels throughout the sequeneF sh hnges re more frequent in dense environments where ojets tend to e very lose to eh otherF sf we use the ttistil model we n set higher flse lrm rte to detet more trgets s shown in the ottom imges of pigure SFPH nd pigure SFPIF e signi(nt mount of flse lrms pper in the prking lots nd t the edges of the uildingsD s these res re responsile for lrge intensity hnges within the pixels in onseutive frmesF 5.4.5 Computational eciency he omputtionl e0ieny of the proposed frmework for multiple ojet deteE tion nd trking depends on three ftorsX

• The number of objectsF he proposed model depends on the numer of ojets in the seneD or more preisely on their density in given reF he denser the ojets re in n reD the more intertions etween the ojetsF husD whenever n ellipse is reted or lolly perturedD the intertions with lrge numer of neighors hve to e omputedF his in turn inreses the omputtionl e'ort during single itertionY

• The size of a frameF he lrger the size of the frmeD the igger the smpling spe nd thusD the more itertions re neededY

• The length of the sequenceF es in the previous seD the longer the seE quenesD the more frmes hve to e proessed nd thusD the longer it tkes until the entire sequene is nlyzedF pirstD the omputtionl e0ieny of the hyrid implementtion of the twgwg smpler with ulmnElike moves is disussed nd ompred to the twgwg smE pler in 2h C s presented in setion SFQFIF henD the performne of the multiple ores implementtion of twgwg is presented nd the speedEup is nlyzedF 5.4. Results

@A

@A @A @dA pigure SFPPX inergy evolution with the numer of itertions for the stndrd twE gwg smpler @greenA nd the twgwg with ulmnElike moves @lueAF he e0E ieny of the stndrd twgwg smpler is signi(ntly inresed y inorporting the properties of sequentil (lters into the irth nd deth perturtion kernelF @AE @A esults on two enhmrk iologil sequenesY @AE@dA esults on the two high resolution stellite imge sequenes with high temporl frequeny from the oulon dt setF 5.4.5.1 RJMCMC using Kalman like moves he irth nd deth kernel using ulmnElike moves e0iently genertes severl irth proposls in single stepF his leds to high irth rte espeilly t the eginning of the simulted nneling proedureF he high temperture t the eE ginning of the nneling proess inreses the eptne proility of otherwise unlikely perturtionsF pigure SFPP depits the energy evolution during the optiE miztion proess when using the ulmnElike moves s opposed to the stndrd irth nd deth kernelsF pigure SFPP @A nd @A show the energy vrition for two enhmrk iologil sequenesD while pigure SFPP @A nd @dA show the energy vriE spatio-temporal marked point process model tion for the two sequenes in the oulon dt setF he energy vlues t the eginning of the optimiztion re slightly higher thn ompred to the stndrd twgwgF he reson for this initil inrese lies in the high numer of irths tht re initilly eptedF he ulmn (lter is used to inrese the overll eptne rtio of the smpler y proposing more meningful perturtionsF roweverD s the temperture in the eginning is very high nd the distriution funtion is similr to uniform distriutionD lrge numer of perE turtions is eptedF es the temperture deresesD the smpler quikly removes dly (tted ojets whih is experiened s rpid drop in the energy vritionF pinllyD towrds the end of the optimiztion proessD mostly nonEjumping trnsforE mtions re performedF sntuitivelyD the irth nd deth kernel using ulmnElike moves o'ers etter nd fster glol explortion of the ojet spe t the eginning of the optimiztionD ompred to stndrd twgwgF es the temperture deresesD the smpler settles in strong lol minimumF owrds the end of the optimiztionD the nonEjumping trnsformtions n e seen s lol djustmentsF he stte vetor of the ulmn (lter ontins not only the lotion nd veloity of the ojetsD ut lso informtion out the size nd orienttion of the ellipseF gonsequentlyD the ulmn (lter genertes proposls of ellipses tht re similr in size nd orienttion nd heneD smller numer of nonEjumping trnsformtions re needed to (t the dtF fy omining the ility to generte multiple irth proposls in single itertion nd more similr ellipses within single trjetoryD the hyrid implementtion of twgwg generlly results in smller numer of itertions needed until onvergene nd thusD n inresed omputtionl e0ienyF st is importnt to notie tht the term 4itertion4 does not diretly relte to the omputtion timeF gonsider for instne the irth of n ojet in the (rst time frme u t 1 F sn the stndrd twgwg settingD irth itertion represents the time needed to propose new on(gurtion X ∪ u t 1 nd ompute the eptne rtio of this perturtionF sn the hyrid smpler settingD the mount of time needed to perform irth itertion using ulmnElike moves n tke ny vlue from the time needed to propose nd ompute the eptne rtio of X ∪ u t 1 to the time needed to propose nd ompute the eptne rtios of

X ∪ u t 1 ∪ u t 2 • • • ∪ u t T F he
ltter se is rehed when the initil proposl X ∪ u t 1 is epted nd then ll susequent perturtions proposed using the ulmn (lter re eptedD exept for u t T for whih the eptne is not mndtoryF reneD it is importnt to nlyze oth the numer of itertions needed until onvergene @pigure SFPPAD ut lso the omputtion time of the two smplers @le SFWAF e di'erent pproh to inresing the e0ieny of the smpler is prllel impleE menttion of the smpling proedureF 5.4.5.2 RJMCMC 2D + T sampler -multiple cores imedding the twgwg smpler into simulted nneling frmework ensures n optiml ury nd t the sme timeD high omputtionl time ost even if pigure SFPQX snrese in omputtionl e0ieny of the proposed multiple ojet trker wFrFtF the numer of ores @g9sAX @leftA for enhmrk iologil imge sequenesY @rightA for the frelon dt set with high temporl frequenyF Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model the properties of sequentil (lters re inorported into the smplerF his n e oserved in le SFW in the se of single proessorF roweverD the redily ville omputtion power @either in the form of omputer lusters or loudsA nd good distriuted design n signi(ntly derese the omputtionl urdenF he mrked point proess models desried in this hpter feture n energy term tht is omputed over the entire sequeneD nmely the lel persistene desried in eqF SFUF husD in distriuted settingD speil re needs to e tken to ensure tht new lels re unique throughout the sequeneF o do thisD the frme numer nd the identity of the omputing unit is used s pre(x for newly ssigned lelF his ensures tht new lels re unique nd do not pper twie in the sme frme whih would violte the mutul exlusion onstrint in etion SFIFIF sn the se of deterministi lelingD the lst itertion is performed in sequentil wy to ensure onsistent leling throughout the sequeneF he prllel implementtion of the smpler proposed in this hpter hs een tested on up to 512 ores on omputer lusterF he omputtion times shown in le SFW nd depited in logrithmi sle in pigure SFPQ revel drsti inrese in the omputtionl e0ieny wFrFtF the numer of ores usedF hese results on(rm tht the proposed implementtion sles very well with the numer of oresF he slility of the proposed method depends on the size of the imge sequeneD on the size of the frme within the sequene nd the numer of ojets present on the seneF husD long sequenes with lrge frmes nd smll numer of ojets re the est ndidtes for the prllel smpling proedure nd exhiit the highest speedEups wFrFtF the numer of oresF 5.4.5.3 Inuence of the perturbation kernels on the convergence speed he omputtion times shown in pigure SFPQ re otined using omintion of spei(lly designed perturtion kernels to inrese the e0ieny of the smplingF st is interesting to evlute the in)uene of the proposed perturtion kernels on the onvergene speedF sn this regrdD we onsider totl of 50 frmes from syntheti iologil sequene hrterized y lrge numer of trgets nd high temporl frequeny @eqF1AF e use the qulity of the rehed energy to evlute nd ompre the performne of twgwg smplers with di'erent perturtion kernelsF he omintions of perturtion kernels re s followsX

• BDM: Birth and death according to a birth mapF his kernel relies on the preEomputtion of irth mps for eh frme of sequeneF he irth mps n e seen s tool tht hnges the underlying homogenous oisson mesure of the point proess into n inhomogeneous oisson mesure sed on some desired propertiesF por instneD we hve omputed the irth mp for frme t s liner omintion of the solute di'erene etween the frme t -1 nd t nd the rdiometri vlues of the pixels in frme tF he frme di'erening is used to identify lotions with lrge hnges from one frme to the next whih is used s n inditor for the presene of moving ojetF he rdiometri vlues within the frme re used to identify possile lotions of 5.4. Results

143

ojets tht do not exhiit ler motion etween frmesF reneD the kernel fvors interesting lotions for the retingGdeleting ojetsY

• BDN: Birth and death in a neighborhood coupled with BDMF his kernel proposes to reteGremove ojets from the urrent on(gurtion in the neighorhood of n existing ojetF his pproh is motivted y the ssumption tht ojets tend to luster together nd heneD it is more likely for ojets to pper lose to other ojets thn to pper totlly isoltedY

• BDML: BDM coupled with local perturbationsF es reminderD exmE ples of lol perturtions re rottionD trnsltion nd sleY

• BDNL: BDN coupled with local perturbationsF his smpler hs een used in etion SFRFSFIY

• BDKF: Birth and death using Kalman-like movesF his hyrid kernel relies on ulmn (lter for the proposl of new ojetsF his pproh is motivted y the proven e'etiveness nd e0ieny of the ulmn (lter for ojet trkingF

• BDKFL: BDKF coupled with local perturbationsF his smpler is used in our experiments in etion SFRFSFIF en inditor of the mixing properties of the smpler is to monitor the eptne rteD iFeF the frtion of proposed perturtions tht re epted @xoteX o not e onfused with eptne rtioF elthough the eptne rte is linked to the eptne rtioD the eptne rtio refers to the proility of epting proE posed perturtionAF sf the eptne rte is lose to 1D then virtully ll proposed moves will e epted t the ost of moving very slowly in the stte speF sf the eptne rte is lose to 0 the hin will virtully not move t llF he hoie of the proposl distriution should im to lne the eptne rte with the onverE gene of the hinF elthough some theoretil results out the optiml eptne rte exist in literture for simpli(ed setups @qelmn et lFD IWWUD oerts nd osenthlD PHHIAD good rule of thum is to hve n eptne rte fr from 0 nd fr from 1F pigure SFPS shows the eptne rte wFrFtF to the temperture prmeter T ∈ [0, 100]D for ll the moves for the smplers presented oveF e temperture prmE eter T ∈ [0, 15] o'ers good proposl distriutions to smple fromF husD in our experiments we set the initil temperture to T = 15 to otin lne etween the eptne rte nd onvergene speedF e wy to ssess the performne of the smplers is to monitor the evolution of the energy of the modelF pigure SFPR shows the energy levels rehed wFrFtF timeF ine the numer of itertions n e misleding in terms of omputtion e0ieny @nd we explined why in etion SFRFSFIAD we hve used the omputtion time s meE sure to evlute the performne of the smplersF he results in pigure SFPR revel two importnt spetsX IF vol perturtions re more e0ient thn deth immeditely followed y irthF his n e oserved oth in the fhxv nd the fhupv smplerF iven though the omputtionl gin from single move is insigni(ntD it eomes pprent over lrge numer of itertionsY PF he ulmnE(lter provides not only signi(nt gin in terms of itertions @s shown in etion SFRFSFIAD ut lso gin in terms of omputtion timesF e hve lso displyed the results for the prllel implementtion of fhxv proposed in etion SFQFS @fheAF elthough omprison etween fhe nd the other smplers would not e firD we only stte the result to give the reder feeling out the tul gin tht n e otined using prllel implementtionsF 5.5 Conclusions wo novel mrked point proess models for the joint detetion nd trking of moving ojets hve een desried in this hpterX ulity model nd ttistil modelF he models rely on three mjor uilding loksX

• e dedited energy funtion hs een de(ned to jointly detet nd trk oE jets in imge sequenesF he proposed models re roust to pperne hngesD illumintion hngesD high displements etween onseutive frmes nd prtil olusions of ojetsF he method hs high tolerne to noiseY

• e powerful proedure for lerning prmeters from the dt sets hs een proposed to tkle the lrge numer of prmeters of the highly nonEonvex energy funtionF he prmeter lerning is posed s onstrined optimizE tion prolem nd liner progrmming is used to serh for fesile solutionY

• en enhned twgwg smplerF wo pprohes to improve the perforE mne of the smpler hve een investigtedF pirstD the twgwg smpler ws lended with the ulmn (lter into hyrid smpler tht ene(ts from the dvntges of othF he good onvergene properties of the twgwg smpler re omined with the e0ieny of the ulmn (lter to produe n e0ient smpler spei(lly dpted to multiple ojet trkingF eondD the prllel implementtion of the twgwg smpler hs een extended to 2h C dimensions nd the good sling properties of this smpler wFrFtF the numer of proessing units hs een shownF ves dé(s dérits iEdessus donnent une idée de l rison pour lquelle l détetion d9ojets dns des imges sttiquesD respetivement l détetion et le suivi d9ojets en mouvement dns des vidéos est très di0ileF gette thèse ne vise ps à relever tous les dé(s mentionnés iEdessusF ves développements réents en déruitge d9imge ho et lFD PHIRD en resturtion d9imge ortill et lFD PHHQD en dé)ouge d9imge hng et lFD PHIQ ou en (ltrge temporel wüller nd wüllerD PHIH fournissent l9étt de l9rt des méthodes permettnt de fire fe à l pluprt des dé(s lors de l9quisitionD du trnsfert et du tritement initil des données stellitiresF gepenE dntD ette thèse orde presque tous les enjeux de l9nlyse d9imge G du vidéo présentées iEdessusF v muvise qulité des données est prise en ompte impliiteE ment gràe à l résistne u ruit du modèle mthémtique utiliséF ne représenttion visuelle des dé(s qui se posent lors de l9nlyse d9imge G de vidéo est présentée dns l pigure fFIF v (gure fFI @A montre deux voitures voisins sur l routeF v modélistion de l9pprene des véhiules est di0ileD r il existe une quntité limitée d9informtionF in outreD u ours du dépssementD les voitures sont très prohes l9une de l9utre e qui rend l détetion individuelle prolémtique pre que les frontières entre véhiules deviennent )ouesF v (gure fFI @A illustre une di0ulté supplémentire usée pr les omresF v9omre d9un grnd âtiment pourrit ostruer totlement un ojet pssnt à trvers ette zoneF v9omre peut églement être générée pr de grndes iles telles que des mionsF gel peut devenir un prolème en prtiulier lorsque plusieurs grnds ojetifs sont situés ensemle dns un groupeF v méthode de détetion et de suivi peut interpréter à tort l9ensemle du groupe omme un unique ojet en mouvementF v (gure fFI @A montre l di0ulté de déteter des ojets en mouvement du fit du mouvement de l mérF ves ouleurs de l9imge montrent le déplement de l mér entre deux trmes onséutivesF n déteteur de mouvement qui ne prend ps en ompte le mouvement de l mér pourrit produire un nomre importnt B.3 Méthodes et approches proposées v9ojetif de ette thèse est d9étudier l9utilistion de ontrintes géométriques dns les méthodes de détetion et de suivi d9ojets vià des proessus pontuels mrquésF gette pprohe permet l9inlusion de deux types de ontrintes géométriquesX l forme de l ile peut être imposée @omme nous tritons priniplement d9ojets friqués pr l9hommeD leur forme peut générlement être e0ement pproximée à l9ide d9une forme prmétrique simpleA et l dispersion géometrique des iles peut être onstrinteF ves proessus pontuels forment une lsse de modèles mthémtiques u sein du thème générl de l géometrie stohstique et ont déjà été ppliqés ve suès à des prolèmes de détetion dns l9imgerie stellitireF hns e qui suitD nous presentons une rève desription de l rison pour lquelle ette lsse de modèles mthémtiques est intéressnte pour le tritement d9imge et de vidéoF @fFSA hns et exempleD tous les pixels u pprtennt à un ojet de l on(gurtion forment l silhoutte de l on(gurtionF errin et lF PHHS proposé d9utiliser les vleurs rdiométriques des pixels pour déterminer si elles pprtiennent à l on(gurtion de l9ojet ou à l9rrièreEplnF in tnt que telD les pixels pprtennt à l on(gurtion ont été modélisés omme une distriution gussienne ve une vleur moyenne élevéeD tndis que les pixels pprtennt à l9rrièreEpln ont été modélisés omme une distriution gussienne ve une vleur moyenne inférieureF ge modèle yésien se heurte à deux di0ultés dns des imges réelles fen rdj et lFD PHIHX IF v lsse de fond n9est en générl ps homogéne et ssumer une distriution gussienne pour tout le fond est peu réliste pour notre prolèmeF r exemE pleD onsidérons une sène urineF i nous voulons déteter des voitures sur l routeD il pourrit être risonnle de supposer que l zone entournt une seule voiture @l routeA suit une distriution gussieneD tndis que l9ensemle de l zone urine ne le fit ps @r et ensemle ontient des omresD des âtimentsD des rresD etFAY PF v proilité des pixels qui pprtiennent à l on(gurtion ne prend ps en ompte les propriétés morphologiques des ojetsF r exempleD si nous onsidèrons une imge ontennt un seul grnd retngle rillnt et si nous souhitons déteter de petites ellipses rillntesD le modèle yésien hoisi onduir à l détetion d9utnt de petites ellipses que possile dns le grnd retngleD e qui n9est ps le résultt souhitéF À l9extérieur de e dre yésienD es limittions ont été dépssées pr une pprohe déteteur dns lquelle le modèle d9oservtion est omposé pr des vrisemlnes loles pour hque ojet fisnt prtie de l on(gurtionF gette pprohe perE mis l9utilistion de proessus pontuels mrqués pour diverses tâhes de détetion d9ojetsF hesmps et lF PHHV ont utilisé un proessus de point mrqué d9ellipses pour ompter le nomre de )mnts roses dns des grndes olonies ontennt des millierrs de spéimensF errin et lF PHHS ont développé un proessus de point mrqué de erles pour l9extrtion des ouronnes d9rresF veur modèles pourE rient églement identi(er l9omre de l9rreD et puis déterminer l position du soleil lorsque l9imge été priseF yrtner PHHR proposé un proessus de point mrqué de retngles pour l détetion de l tre u sol des âtimentsF ves intertions d9lignement entre les retngles ont été dé(nis (n de reproduire l disposition des âtimentsF lus réemmentD fen rdj et lF PHIH ont introduit un proessus de point mrqué d9ellipses pour ompter les teux dns des portsF r onséquentD les modèles de proessus pontuels ont montré leur potentiel pour l détetion d9ojets dns des imges hute résolutionF B.3.2 Application à la détection et suivi des objets dans des vidéos v9ojetif de ette thèse est d9étudier l possiilité d9utiliser des modèles de proE essus de point mrqué pour l détetion et le suivi d9ojets en mouvement dns des vidéosF v détetion et le suivi d9ojets en mouvement jouent un rôle imporE tnt dns l9nlyse vidéo où il représentent une étpe néessire dns l rélistion d9nlyses de hut niveuD pr exemple l surveillne terrestre et mritime du trfE (F ve prolème de l détetion d9ojets dns des imges sttiques est églement ordé dns ette thèseD ve un fort ent sur le prolème prtiulièrement di0ile de l9extrtion des teux dns des portsF ves priniples données utilisées dns ette thèse sont des imges stellitires à très hute résolution et des vidéos qui ont été fournies pr eirus hefense nd peF ges ensemles de données vrient à l fois en tille et fréquene temporelleD llnt de jeux de données ve peu de trmes à des fréquenes temporelles sses @14 trmes à 1 -2 rzA jusqu9à de longues vidéos à de hutes fréquenes temporelles @plus de 3000 trmes à 30 rzAF ves informtions de vérité terrin pour es ensemles de • Quality modelF ge modèle omine une preuve de ile qui est lulée en fontion de l di'érene de trmes onséutives pour lquelle un seuil est ppliqué en mesurnt l distne de ontrst entre l9intérieur et le ord de l9ellipseF v di'érenition des imges est utilisée pour identi(er les endroits suseptiles de ontenuir des iles moilesF ve résultt du dre de di'érenE ition est une imge en niveux de gris dns lequel l9intensité d9un pixel représente l quntité de hngement entre les trmes présente à et endroitF lus l quntité de hngement @respF plus l9intensité d9un pixel est grndeAD plus il est prole pour une ile moile soit présente à et endroitF ne mesure de distne de ontrste est lors utilisée pour 0ner l détetion et extrire des informtions telles que l9orienttion et l tille des ilesF

• Statistical modelF gette méthode repose sur l théorie de l déision yésiE enne et onsiste à tester sttistiquement deux hypothèses et hoisir l plus vrisemlleF hns notre ontexteD l9ojetif est de déterminer si une ile est présente ou senteF out d9ordD nous lulons l di'érene entre deux trmes onséutivesF e(n de déterminer l proilité d9existne d9une ileD nous utilisons une fenêtre glissnte que nous trnsltons d9un pixel à l foisF xous onsidérons les deux hypothèses suivntesX H 0 X l9ellipse ne ouvre que l9rrièreEpln uune ile n9étnt présenteY H 1 X l9ellipse est plé dns le entre d9une ileF xous utilisons l règle de déision de xeymnEerson pour hoisir entre les deux hypothèses H 0 et H 1 F ne desription détillée de es deux modèles énergétiques externes peut être trouvée dns l prtie SFIFPF Terme d'énergie totaleF ve terme d9énergie totle peut être érit omme l somme de tous les termes d9énergie dé(nis iEdessusX

• Quality modelX U θ (X, Y) = γ dyn U int dyn (X) + γ label U int label (X) + γ o U int overlap (X)+ γ ev U ext ev (X|Y) + γ cnt U ext cnt (X|Y). @gFRA • Statistical modelX U θ (X, Y) = γ dyn U int dyn (X) + γ label U int label (X) + γ o U int overlap (X)+ γ stat U ext stat (X|Y).
@gFSA où X est l on(gurtion d9ellipsesD Y sont les données d9imges et γ • sont les poids ssoiés à hque terme de l9énergie totleF n totl de inq @respF qutreA poids γ • sont néessires pour équilirer les termes individuels dns l9énergie (nle du ulity model @respF ttistil modelAF ges prmètres ne peuvent ps être réglés de mnière empiriqueF ves méthodes existntes d9estimtion des prmètres fondés sur l9lgorithme d9ispérneE wximistion @voir hempster et lF IWUUA sont très longues et générlement instles dns e sF r onséquentD nous vons développé une tehnique lterntive d9pprentissge des prmètresD (n d9méliorer l stilité du modèle et de réduire l hrge de lulF Estimation des paramètresF ne propriété lé de l9énergie dérite dns l9éqF gFR et l9éqF gFS est s linérité pr rpport ux prmètres de poidsF gette propriété est ruile pour déterminer un lgorithme d9pprentissge ppropriéF v linérité est exploité dns l etion SFPF en proposnt une proédure d9pprentissge des prmètres fondée sur l progrmmtion linéireF h9une prtD l fontion de densité posteriori π(X) est une ominison linéire des prmètres de poids pour toute on(gurtion donnéeD XF h9utre prtD si nous utilisons le twgwg pour simuler les deux modèles proposésD seul le rpport π(X )/π(X) doit être lulé dns une trnsition de l hîne de wrkov sns onnître l vleur exte de π(X ) ni de π(X)F e prtir de es propriétésD un ensemle de ontrintes π(X )/π(X) ≥ 1 @or ≤ 1A peut être étliD si une on(gurtion est onnue pour être meilleur q9une utreF ges ontrintes peuvent ensuite être trnsformées en inéglités linéires des prmètres de poidsF in(nD prés voir reueilli un nomre su0snt d9inéglités linéiresD l progrmmtion linéire peut être utilisée pour trouver une solution rélisle pour les prmètres de poidsF ne fois que les prmètres du modèle ont été trouvésD l fontion de densité peut être simulée pour déterminer l meilleure on(gurtion d9ojets qui dérivent un ensemle de données donnéF SimulationF ves énergies dérites dns l9éqF gFR et l9éqF gFS ne sont lirement ps onvexeF sl est file de onstruire des exemples qui ont deux minim prtiqueE ment éguxD séprés pr un mur de vleurs élevées de l9énergieF v dépendne usée pr les ontrintes physiques d9ordre élevé est l priniple rison qui pousse l9énergie à être nonEonvexeF v distriution ilée est l distriution postérieure XD iFeF π(X) = f (X|Y)D dé(nie sur une union de sousEespes de dimensions di'érntesF v méthode d9optimistion l plus lrgement onnue pour les fontions d9énergie nonEonvexes et un nomre inonnu d9ojets est le twgwg @ussi ppelé wgwg à sut réversileA développé pr qreen IWWSF ve twgwg utilise un mélnge de noyux de perturtion pour réer des tunnels à trvers les murs de hute énergieF xous utilisons le reuit simulé pour trouver un minimiseur de l fontion de l9énergieF v fontion de densité dérite dns l9éqF gFI peut être réérite ommeX gette étude nous permis d9évluer le potentiel des modèles de proessus de point mrqué pour le suivi d9ojetsF xous vons montré que es modèles sont de puissnts onurrents pr rpport à l9étt de l9rt eu égrd à l qulité des résulttsF ourtntD des méliortions onstntes doivent être fites onernnt l proédure d9optimistion (n de fire des modèles de proessus de point mrqué un hoix de premier pln dns l9venirF ve vste trvil de reherhe tuellement mené dns le monde sur l détetion et le suivi d9ojets dns des imges stellitires insi que dns des imges de miE rosopie un impt mjeur sur notre vie quotidienneF he l séurité rue et l surveillne de l fune à l9nlyse de l9tivité suEellulireD l détetion et le suivi d9ojets dns es domines sont l9un des los de onstrution d9une ompréhension plus profonde de notre monde et des motifs qui le régissentF Bibliography F egrwl nd hF othF verning sprse representtion for ojet detetionF Proc. ECCVD pges IIQ!IQHD PHHPF @gited on pge PQFA qFF endrewsF Foundations of multithreaded, parallel and distributed programmingF Abstract: hns ette thèseD nous ominons les outils de l théorie des proilités et de l géométrie stohstique pour proposer de nouvelles solutions u prolème de l détetion et le suivi d9ojets multiples dns des séquenes d9imges hute résolutionF out d9ordD nous développons un modèle de proessus pontuel mrqué sptil pour déteter une lsse prédé(nie d9ojets @teuxA en fontion de leurs rtéristiques visuelles et géométriquesF xous onevons une mise en oeuvre de multiEoeur de l9éhntillonneur de wgwg à suts réversilesF xous montrons des résultts supérieurs à eux fournis pr des methodes proposées dns l9étt de l9rt et des temps de détetion ompétitifsF ves très ons résultts otenus nous ont motivés pour étendre es modèles dns le domine temporel et pour réer un dre fondé sur des modèles de proessus pontuels mrqués sptioEtemporels pour déteter et suivre onjointement plusieurs ojets dns des séquenes d9imgesF xous proposons l9utilistion de formes prmétriques simples pour dérire l9pprition de es ojetsF xous onstruisons de nouveux modèles fondés sur des énergies dédiées onstituées de plusieurs termes qui tiennent ompte à l fois l9tthe ux données et les ontrintes physiques telles que l dynmique de l9ojetD l persistne de l trjetoire et de l9exlusion mutuelleF xous onstruisons un shém d9optimistion pproprié qui nous permet de trouver des minim loux de l9énergie hutement nonEonvexe proposée qui soient prohe de l9optimum glolF gomme l simultion de es modèles requiert un oà t de lul élevéD nous portons notre ttention sur les dernières mises en oeuvre de tehniques de (ltrge pour le suivi d9ojets multiplesD qui sont onnues pour à a tre moins oà teuses en lulF xous proposons un éhntillonneur hyride ominnt le (ltre de ulmn ve l9éhntillonneur wgwg à suts réversilesF hes tehniques de lul de hute performne sont églement utilisées pour ugmenter l9e0ité de lul de notre méthodeF xous fournissons une nlyse en profondeur du dre proposéD insi qu9une omprison extensive ve des proédés de l9étt de l9rt sur l se de plusieurs métriques lssiques de suivi d9ojets et de l9e0ité de lulF gette nlyse montre une très onne performne de détetion et de suivi insi que d9une omplexité rue des modèlesF hes tests exhustifs ont été menés à l fois sur des séquenes d9imges stellitires hute résolution et sur des données de mirosopieF Mots-clés: uivi d9ojets multiplesD détetion d9ojetsD proessus pontuel mrE quéD (ltre de ulmnD séquenes d9imges stellitiresD séquenes des données de mirosopieD hute résolutionF

f θ,i (X = X|Y) = 1 c T emp i (θ|Y) exp - U θ (X,
Stochastic geometry for automatic multiple object detection and tracking in remotely sensed high resolution image sequences Abstract: sn this thesisD we omine the methods from proility theory nd stohsti geometry to put forwrd new solutions to the multiple ojet detetion nd trking prolem in high resolution remotely sensed imge sequenesF pirstD we develop sptil mrked point proess model to detet preEde(ned lss of ojets @iFeF otsA sed on their visul nd geometri hrteristisF e design multiple ore implementtion of the reversile jump wgwg smplerF e show improved results over stte of the rt methods nd ompetitive detetion timesF he very good results otined motivted us to extend these models to the temporl domin nd rete frmework sed on sptioEtemporl mrked point proess models to jointly detet nd trk multiple ojets in imge sequenesF e propose the use of simple prmetri shpes to desrie the pperne of these ojetsF e uild newD dedited energy sed models onsisting of severl terms tht tke into ount oth the imge evidene nd physil onstrints suh s ojet dynmisD trk persistene nd mutul exlusionF e onstrut suitle optimiztion sheme tht llows us to (nd strong lol minim of the proposed highly nonEonvex energyF es the simultion of suh models omes with high omputtionl ostD we turn our ttention to the reent (lter implementtions for multiple ojet trkingD whih re known to e less omputtionlly expensiveF e propose hyrid smpler y omining the ulmn (lter with the stndrd eversile tump wgwgF righ performne omputing tehniques re lso used to inrese the omputtionl e0ieny of our methodF e provide n inEdepth nlysis of the proposed frmeworkD s well s n extensive omprison with stte of the rt methodsD sed on stndrd multiple ojet trking metris nd omputtionl e0ienyF his nlysis yields very good detetion nd trking performne t the prie of n inresed omplexity of the modelsF ixhustive tests hve een onduted oth on high resolution stellite nd mirosopy imge sequenesF Keywords: wultiple ojet trkingD ojet detetionD mrked point proessD ulmn (lterD stellite imge sequenesD mirosopy dt sequenesD high resolutionF

•

  Artifacts re rti(il strutures tht re ontined within the imge nd represent perturtion of the signlF ixmples of rtifts in this step inlude sensor sturtion or eGh @nlogGdigitlA onversion proE lems qonzlezD PHIQF • Blurred images re result of fst sensor G ojet motion nd is prevE lent in wek illumintion settings whih led to long exposure times of the merY • Weak contrast ppers due to environmentl onditionsF pei( wether onditions suh s fog or louds re ommon situtions whih led to wek ontrst in optil mersY PF Image / video transfer • Artifacts during this step my e used y distured onnetion whih n led to strong rtifts or even missing imges qonzlezD PHIQY • Compression-decompression artifacts re lokElike strutures n signi(ntly degrde the qulity of the dt xetrvli nd rskellD IWWSD entoniniD PHHQY QF Image / video analysis

  his de(nition n esily e extended to video y onsidering 3Edimensionl spe suh tht for eh pixel uD u ∈ K × T ⊆ Z 3 D where K represents the support of n imge nd T is the time xisF por simpliityD we will only present the rtionle 1.3. Approach and proposed methods in the se of 2Eh imgeF vet (Ω, A, P) e proility speD then stohsti model of n imge onsists in onsidering the gry vlues of eh pixel to e reliztion of stohsti version of the funtion IF I is now rndom vrileXI : Ω → R K .@IFIA e proility lw P I is ssoited to the rndom vrile I suh tht P I (A) = P(I -1 (A))F he simplest imge model onsists of ll pixels eing independentXP(I ∈ A) = u∈K P(I(u) ∈ A u )F prom physilpoint of viewD n imge n e onsidered s noisy oservtion of n underlying phenomenonF en e0ient wy to pture this physil reltion is to use fyesin frmework to model the imgeF e fyesin model of n imge n e written sX P(I = y ∈ A) = x P(I = y ∈ A|X = x)P(X = x) @IFPA where P(I = y ∈ A) is the mrginl lw of the oservtionsD P(I = y ∈ A|X = x) is the lw of the oservtion nd P(X = x) is the priori lw whih inorportes prior knowledge on the underlying phenomenon XF sing fyes9 theorem we n nlyze the phenomenon X strting from oservtions y 1 , • • • , y n s followsX P(X = x|I = y) ∝ P(I = y|X = x)P(X = x). @IFQA he fyes model is interesting from sttistil point of view s it provides lss of nturl estimtorsX the fyes estimtors wrinD PHHUD qelmn et lFD PHIRF yne suh estimtor is the wximum e osteriori @weA riterion whih n e written sX XMAP = arg max x n i=1 P(X = x|I i = y i ) @IFRA when ll oservtions y 1 , • • • , y n re independentF

1. 4

 4 Thesis organization and contributions he thesis is orgnized s followsX • Chapter 1 provides n overview of the thesisD desries the dt usedD outlines the motivtion of this work nd summrizes the ontriutionsY • Chapter 2 reviews stte of the rt pprohes to the prolems of ojet detetion nd ojet trkingF • Chapter 3 presents point proesses s suitle frmework for ojet deteE tionD ounting s well s trkingF tte of the rt prmeter estimtion nd optimiztion tehniques spei(lly dpted to this frmework re desriedF • Chapter 4 introdues the (rst importnt ontriutions of this thesisF st provides the model developed for ot detetion nd ounting in hror resF he ontriutions of the hpter n e resumed s followsX etion 4.1. presents n improved mrked point proess model developed for ot detetion nd ompres it to the model previously introdued y fen rdj et lF PHIHY etion 4.2. desries the prmeter estimtion tehniques used to set the prmeters of the modelY etion 4.3. proposes new prllel implementtion of the widely known twgwg optimiztion shemeF • Chapter 5 introdues the min ontriution of this thesisF sn this hpter we desrie mrked point proess model spei(lly developed for multiple ojet trkingF he ontriutions of the hpter n e resumed s followsX etion 5.1. introdues the new model nd presents n inEdepth desripE tion of the onstrints nd restritions of this modelY etion 5.2. develops novel liner progrmming pproh to estimte the prmeters of the model in the point proess frmeworkY 20 Chapter 1. Introduction etion 5.3. proposes new perturtion kernels spei(lly djusted for the prolem of multiple ojet trking to e used in the twgwg optimiztion shemeF • Chapter 6 onludes the thesis nd outlines the perspetives of this workF 1.5 Publications he following ppers hve een pulished or sumitted sed on works ontined in this mnusriptX • Journal papersX F gr iunD wF yrtnerD nd tF eruiF e sptioEtemporl mrked point proess model for joint detetion nd trking of moving ojetsF uE mitted to IEEE Transactions on Image ProcessingD PHISY F gr iun nd tF eruiF nsupervised mrked point proess model for ot extrtion nd ounting in hrors from high resolution optiE l remotely sensed imgesF sn Revue Francaise de Photogrammétrie et TélédétectionD volF PHUD ppF QQERRD PHIRY • National and International conferencesX F gr iunD wF yrtnerD nd tF eruiF umitted to siii gomputer ision nd ttern eognitionD eD PHITY F gr iun nd tF eruiF umitted to siii snterntionl ymposium on fiomedil smgingD gzeh epuliD PHITY F gr iunD wF yrtnerD nd tF eruiF toint detetion nd trking of moving ojets using sptioEtemporl mrked point proessesF sn IEEE Winter Conference on Applications of Computer VisionD eD PHISY F gr iun nd tF eruiF owrds e0ient simultion of mrked point proess models for ot extrtion from high resolution optil remotely sensed imgesF sn International Geoscience and Remote Sensing Sympo-siumD gndD PHIRY F gr iun nd tF eruiF nsupervised mrked point proess model for ot extrtion in hrors from high resolution optil remotely sensed imgeF sn International Conference in Image ProcessingD eustrliD PHIQY F gr iunD wF yrtnerD nd tF eruiF sntegrting twgwg nd ulmn pilters for multiple ojet trkingF sn GRETSI -Traitement du Signal et des ImagesD prneD PHISY F gr iun nd tF eruiF fot extrtion in hrors from high resoluE tion stellite imges using mthemtil morphology nd mrked point proessesF sn GRETSI -Traitement du Signal et des ImagesD prneD PHIQF Chapter 2 State of the art

  @QFSVA sn his lultion yrtner PHHR uses two hypothesesX 52 Chapter 3. Using Point processes for object detection ξ(•) is di'eomorphismD whih implies tht Σ hs the sme dimension s ∂(u)D nd Σ nd W hve lso the sme dimensionY the lw of Z is dominted y the veesgue mesure nd denotes f Z (•)

  otherE wiseY tionl urden hs motivted mny reserhers to investigte other lgorithms to e'etively nd e0iently simulte mrked point proessesF uh lgorithms will e presented lter in this setionF yn the other hndD the rpid development of tehE nology led to new softwre implementtion prdigmsF xowdysD the trditionl singleEproessor sequentil omputing is repled y fst ped multiple ore prlE lel omputing prdigmsF his tehnologil evolution inspired erdié nd vfrge PHIP to propose prllel implementtion of the twgwgF erdié nd vfrge PHIP showed tht this implementtion leds to signi(nt redution in omputE tion time in the se of imge nlysisF 3

  otsF e (rst model to detet ots in hrors in optil stellite imges ws developed y fen rdj et lF PHIHF he uthors pled three strong onstrints on the position nd orienttion of otsD s followsX IF ll ojets must hve the sme orienttionY PF ojets should e tngent to eh other nd their enters should e lignedY QF ojets must not overlp more thn given extentF harbors pigure RFIX veftX rtiulr se of hror where ll ots hve the sme orienttionF ightX qenerl se of hrors where ots hve di'erent orienttionsD sed on their position in the hrorF houghD this model might e dequte for prtiulr ses of hrorsD the (rst two onstrints re too restritive for the generl seF pirstD the orienttion of ots in hror vriesD s shown in pigure RFIF eondD the size of neighoring ojets n di'er nd thusD their enters will e shiftedF sn this hpterD new model is developed tht hndles generl ses suh s the ones previously presentedF he ssumptions mde on the ojets re the followingX IF the orienttion of eh ot is determined lollyY PF neighoring ots should hve similr orienttionY QF ots should not overlp more thn given extentF he (rst ssumption implies tht the orienttion of ots is determined lolly y using ues ville in the imgeF he seond ssumption restrits neighoring ots from hving ompletely di'erent orienttions whih is generlly true in hrorsF pinllyD s in the model of fen rdj et lF PHIHD the mount of overlp etween ojets should e kept reltively smllF 4.1 Model gonsider mrked point proess of ellipsesF he ojet speD WD is ounded set in R 5 de(ned sX

  the width nd height of the imge yD respetivelyD [a m , a M ] is the rnge for the length of the semiEmjor xisD [b m , b M ] is the rnge for the length of the semiEminor xis nd ω ∈ [0, π] is the orienttion of the ellipseF he imge oservtion y = [0, IW M ] × [0, IH M ] is PEdimensionl rry of IW M × IH M pixelsF es we onsider only gryEsle imgesD the vlue of pixel elonging to y is given y rel numerF his model is prtiulr se of the oservtion model 4.1. Model 65

  representtion of overlpping ellipses wFrFtF (xed overlpping rtio s = 0.5F he energy of the on(gurtion inreses s the mount of overlpping etween ojets inresesF e disussed in setion RFPF pinllyD

  alω l (x). @RFITA he prmeters of the model re desried y the prmeter vetor θ = [θ ext , θint] = [γ cnt , γ c , d 0 , ρ, s, γ al ]F he next setion presents detiled desription of how eh of these prmeters re determinedF 4.2 Parameter estimation wodeling n imge using mrked point proess usully results in hevy model with lrge numer of prmetersF sn this setionD the most importnt prmeters of the previously desried model re presented in more detilF es reminderD the prmeter vetor is θ = [γ cnt , γ c , d 0 , ρ, s, γ al ]D where • γ cnt is the weight of the externl energy term wFrFtF the internl energy termY

  d 0 (y) does not hve signi(nt e'et on the overll externl energy termF yn the other hndD Chapter 4. Spatial marked point process model for boat extraction in harbors if γ c ≥ 1.5D the overll externl energy term for true positives inresesF husD good vlues for γ c re γ c ∈ [0.7, 1.4]F 4.2.3 Determining the threshold d 0 he threshold d 0 for the ontrst distne mesure is determined using heuristi pprohF ine the imges onsidered ontin mostly wter nd the ojets of interest @iFeF otsAD the pixels re roughly divided into two lsses in the following wyX IF ghoose n initil thresholdD D i=0 = D 0 F sullyD D 0 is tken to e the middle of the greyElevel rngeY PF egment the imge into two lssesD ojets nd kgroundD s followsX • ojet lssX C o = {I(m, n) : I(m, n) > D}D where I(m, n) represents the pixel greyElevel t lotion (m, n) in the imge IF • kground lssX C b = {I(m, n) : I(m, n) ≤ D}F QF gompute the verge vlues of the two lsses suh tht m 1

  δ b ]F e sle perturtion modi(es the length of the semiEmjor nd 4.3. Optimization 77 semiEminor xis of n ojet u s followsX

•

  st n hndle drk spots inside the ojets through the use of n internl order to ompute the ontrst distne wFrFtF the externl orderF husD the drk spots within the ots hve less in)uene on the overll ontrst distne used in the dt likelihoodY• st n hndle ojets of vrying sizes through the use of lrge intervl for Chapter 4. Spatial marked point process model for boat extraction in harbors the semiEmjor nd semiEminor xis of the ellipses used to extrt the otsY• st n hndle ojets with di'erent orienttions through the orienttion onE strint used in the internl energy termF he generl heuristi wFrFtF the orienttion of the ots in hror is tht they tend to e nhored perpenE diulr to the keysF he orienttion onstrint present in the model fvors ellipses tht re oriented perpendiulr to the wter res tht re formed etween the keys of the hrorF he model hs een used to detet ots in two types of imgesX pot 5 imgesD tken over the prenh ost nd provided y gxiY nd leides imgesD tken over welourneD eustrli nd provided y eirus h8F he detetion ury for the pot 5 imges is goodF wissed detetions pper minly due to the proximity of the otsD whih dereses the ontrst distne in the externl energy term nd thusD mkes ojets less likely to e detetedD ut lso due to the low rdiometri vlue of the ot itselfF plse lrms pper minly on the keys tht hve high rdiometri vlueF he omputtionl e0ieny of the proposed model inreses s the numer of proessors used for the optimiztion inresesF he omplexity of the model leds to the neessity of higher omputtionl e'orts wFrFtF other ojet extrtion methodsD ut the extrtion ury ompenstes for the omputtionl urdenF his model might not e desirle in the se when only rough estimte of the lotions of the ojets of interest is requiredF xeverthelessD the model proves to e superior when n ext ssessment of the sene is neessryFChapter 5

  Contents 5.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.1.1 Internal energy term . . . . . . . . . . . . . . . . . . . . . . . 92 5.1.2 External energy term . . . . . . . . . . . . . . . . . . . . . . 95 5.1.3 Total energy term . . . . . . . . . . . . . . . . . . . . . . . . 102 5.2 Parameter estimation . . . . . . . . . . . . . . . . . . . . . . . 102 5.2.1 Linear programming . . . . . . . . . . . . . . . . . . . . . . . 102 5.2.2 Weight estimation of individual energy terms as a linear programming problem . . . . . . . . . . . . . . . . . . . . . . . . 104 5.3 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.3.1 RJMCMC in 2D + T . . . . . . . . . . . . . . . . . . . . . . 107 5.3.2 Proposition kernels used for object tracking . . . . . . . . . . 109 5.3.3 Consistent labeling . . . . . . . . . . . . . . . . . . . . . . . . 112 5.3.4 Integrating Kalman like moves in RJMCMC . . . . . . . . . 116 5.3.5 Ecient implementation of RJMCMC in 2D + T -multiple cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 5.4.1 Tracking results on synthetic benchmarks used by the biological processing community . . . . . . . . . . . . . . . . . . . . 122 5.4.2 Tracking results on real biological data . . . . . . . . . . . . . 125 5.4.3 Tracking results on simulated low temporal frequency satellite data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 5.4.4 Tracking results on simulated high temporal frequency satellite data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.4.5 Computational eciency . . . . . . . . . . . . . . . . . . . . . 138 5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145sn the previous hpter the use of mrked point proesses for ojet detetion hs een investigtedF e point proess of ellipses ws used to detet ots in hrE ors whih is hllenging se due to the prtiulr distriution of the ojetsF 90 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model xeverthelessD the detetion results were very promising with high ury levels nd oth good preision nd rell rtesF his hpter revolves round the following questionX row n mrked point proE ess models e suessfully pplied to the joint detetion nd trking of multiple ojetsc he word 9suessfully9 should e interpreted not only s merely solving the trking prolem using suh proessesD ut doing so with high ury nd e0ieny rtes whih re ompetitive with stte of the rt methods in the omputer vision (eldF upplementry to the ojet detetion prolemD ojet trking involves new hllenge in the form of ssoiting individul detetions with one ojetF he inE dependent detetions of single ojet throughout the imge sequene hve to e grouped together into meningful trks in order to infer the trjetories nd motion hrteristis of tht ojetF his dditionl tsk n eome prtiulrly di0ult in dense res with multiple ojets rossing their pths or eing very lose to eh otherF his hpter gives the nswer to the question rised ove nd sustins it with detiled nlysis nd onvining exmplesF he hpter is orgnized s followsX (rstD mrked point proess model proposed for the joint detetion nd trking is presentedF he pplitionEdependent prmeters of the model re identi(ed nd proedure for lerning these prmeters is disussedF xextD optimiztion methods re investigted nd new optimiztion proedure spei(lly dpted for trking is proposedF pinllyD results re shown on lrge vriety of imge sequenes nd the performne of the novel method is disussed in detilF gonlusions re drwn t the end of this hpter wFrFtF the suitility of mrked point proess models to the joint detetion nd trking of multiple ojetsD their dvntges nd their drwksF 5.1 Model gompred to stti imgeD n imge sequene hs n dditionl dimensionX the dimension of timeF sn the ontext of mrked point proessesD time n e modeled in t lest two oneptully di'erent wysX • Time as a markF he point proess n e viewed s ounting proess de(ned on the 2h speF he time mrk ould represent the numer of pE pernes of n event in given lotionF sn the se of stellite imgesD this pproh n e e0iently used for pplitions suh s tr0 density estimtionF • Time as a dimensionF sn this seD the point proess exists in the 3h spe @two sptil dimensions nd temporl oneAF ptil s well s temporl orE reltions n e expliitly desried in this pprohD whih mkes it suitle for trking purposesF reneD time is modeled s n dditionl dimension in this workF he 3h imge ue is modeled s ounded set K = [0, I hmax ] × [0, I wmax ] × {1, • • • , T }F e mrked point proess of ellipses is onsidered on KD with the mrk spe M de(ned s M = [a m , a M ] × [b m , b M ] × (-π 2 , π 2 ] × [0, L]D where a m D a M nd b m D b M re the minimum nd mximum length of the semiEmjor nd semiEminor xis respetivelyD

  5.1.1 Internal energy term he internl energy term onsists of set of onstrints ment for orret detetion of ojets nd to filitte trkingF hese onstrints trget the lyout of the 5.1. Model 93 ojetsF por instne geometri nd physil onsisteny should e mintined @eFgF interpenetrtion mong ojets should e voidedAF 5.1.1.1 The dynamic model e de(ning property of trking @s opposed to individul detetions per frmeA is tht in most ses ojet trjetories re smoothF his llows to fvor on(gurE tions where ojets exhiit motion desried y dynmi modelF his motion modelD denoted y d(•, •, •)D depends on the pplitionF he dynmi model used throughout this work is onstnt veloity modelF vet v ∈ x t-1 D u ∈ x t D w ∈ x t+1 e three ojetsD then the dynmi model n e written in terms of their positions pos(v)D pos(u) nd pos(w) s followsX d(v, u, w) := ||pos(v) -2pos(u) + pos(w)|| 2 @SFRA xote however tht ny motion model n e inserted into the modelF xeverthelessD the onstnt veloity model is generl enough to over lrge vriety of trking pplitionsF en energy term is designed to fvor ojets whih follow given motion model sFtF for n ojet u tht exists t time t it is written sX

  the numer of pixels in the res I u D F u nd A respetivelyY E σ2 Iu D σ2 Fu D σ2 A re the estimted vrines of the res I u D F u nd A respetivelyF es we hve mde the ssumption tht σ 2 Iu = σ 2 Fu = σ 2 A = σD we n further simplify eqF SFPT toX log Λ = N Iu mIu + N Fu mFu -N A mA , @SFPUA whereX E N Iu D N Fu D N A re the numer of pixels in the res I u D F u nd A respetivelyY E mIu D mFu D mA re the estimted mens of the res I u D F u nd A respetivelyF 5.1.2.2.3 Choosing a threshold τ e hve otined simple nd reltively fst preEdetetion lgorithm tht n e desried in the following wyX por eh time instne tD for eh pixel of the di'erene imge F D t D ompute the generlized likelihood rtio given y eqF SFPU for eh sliding ellipse uF por eh time tD we otin 2h mtrix whih we deE note with M t GLRT F e threshold τ is pplied to this mtrix sed on the desired peF 100 Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model pigure SFRX yg urves for the syntheti iologil dt set with 3 di'erent window sizesX 9 × 9 @redAD 11 × 11 @lueA nd 15 × 15 @greenAF he yg urves show tht window size of 9 × 9 gives the est resultsF pigure SFSX veftX yntheti iologil imge ontining 32 trgetsF widdleX he output of M t GLRT F ightX reEdetetion resultsF pigure SFTX veft topX tellite imge of oulon ontining 2 trgets @otsAF veft ottomX reEdetetion resultsF he flse lrms re used y wves hitting the shoreD resulting in high hnges etween onseutive frmesF ightX he output of M t GLRT F 5.1. Model 101 pigure SFUX he e'ets of di'erent omponents of the energy termsF he upper row shows on(gurtion with higher energy vlue for eh individul termF he ottom row shows on(gurtion with lower energy vlue for eh individul termF he drk spots denote trget lotions t di'erent time frmesF hi'erent olors on the trgets represent di'erent lels ssigned to ehF

•

  Birth and death in a neighborhoodX this kernel is used to propose the ddition or removl of n interting pir of ojetsF o de(ne the neighorhood of n ojet we introdue the notion of event conesF his notion ws previously introdued y veie et lF PHHV to serh for plusile trjetories in the speEtime volume y linking up event onesF pollowing the ide of veie et lF PHHVD the event one of n ojet u = (c h , c w , t, a, b, ω, l) is de(ned s depited in pigure SFW @dAD to e the speEtime volume it n physilly in)uene from its urrent positionY • Local transformationsX lol trnsformtions re trnsformtions tht rnE domly selet n ojet u in the urrent on(gurtion nd then propose to reple it y pertured version of the ojet vX X = (X \ u) ∪ vF rnslE tionD rottion nd sle re exmples of suh trnsformtionsF e mpping R m (•, •) : C × C → (0, ∞)D lled the qreen rtio @or eptne rtioAD is ssoited to eh of these perturtion kernelsF et itertion iD the proposition

  where (x u , y u ) is the ojet9s lotion in speD t u is the time frme in whih the ojet existsD (a u , b u , ω u ) re the semiEmjor xisD semiE minor xisD respetively the orienttion of the ellipse nd l u is the lel ssoited to the ojet whih is used to extrt the trjetoriesF • Birth and death using a birth mapF e irth mp is mpping b(•) tht ssigns to eh pixel in frme the proility of tht pixel ontining the enter of n ellipseF ih frme in sequene hs its ssoited irth mp b t (•)F he irth mps trnsform the homogeneous oisson intensity mesure Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model ν(•) into n inhomogeneous oneD sed on the lotion nd time frmeF reneD ojets re no longer hosen uniformly in W F he kernel n e written s Q BDM (X,•) = p b (X)Q BM (X, •) + p d (X)Q DM (X,•)D where p b = p d = 0.5 represent the proility of irth @respF dethA nd Q BM (X, •) is the irth suEkernel whih retes new oE jets sed on the ssoited irth mps nd Q DM (X, •) deletes n ojet ordinglyF he qreen rtios n e written s followsX Birth:

  D where ∂(u) desries the neighorhood of ojet u ording to symmetri reltion ∼ t whih is desried in eqF SFRQY QF ropose X = X ∪ vF o generte new ojet v in reltion ∼ t with uD we use rndom vetor z on the spe Σ ording to the lw of rndom vrile Z nd pply n injetion ξ u (•) to the otined result s presented in etion QFQFIFRX

  the set of ll mximl trjetories tht exist within on(gurtion XF purthermoreD let T d l e symmetri trnsformtion suh tht d l ∈ [0, L] nd T d l hnges the lel of n ojet u s followsX

  B.3.1 Probabilités dans l'analyse des images / du vidéo hns l9nlyse d9imgeD les proessus pontuels mrqués sont une évolution nturelle des hmps de wrkov en rison de l9ugmenttion signi(tive de l résolution sptile des imges du fit des réent pteursF ve leteur intéressé pr une nlyse détillée de ette évolution peut se référer ux mnusrits d9rd de érez PHHQ et hesomes PHHRF B.3.1.1 Un modèle stochatisque d'une image ne imge est une ensemle de pixelsF ne imgeD notée ID peut être représente pr une fontion qui ssoie une vleur de niveu de gris à hque pixel uD u ∈ K ⊆ Z 2 XI : K → R u → I(u).gette dé(nition peut filement être étendue pour une vidéo en onsidérnt un esE pe en 3 dimensions tel que pour hque pixel uD u ∈ K × T ⊆ Z 3 D où K représente le support d9une imge et T est l9xe du tempsF our plus de simpliitéD nous ne présenterons que l justi(tion dns le s d9une imge en 2EhF oit (Ω, A, P) une espe de proilitéD un modèle stohstique d9une imge onE siste à onsiderér que les vleurs de viveu de gris de hque pixel peuvent être omme une rélistion d9une version stohstique du fontion IF I est mintennt une vrile létoireXI : Ω → R K . @fFIAne loi de proilité P I est ssoié à l vrile létoire I telle que P I (A) = P(I -1 (A))F ve modèle d9imge le plus simple onsiste à onsiderer tous les pixels omme onditionnellement indépendntX P(I ∈ A) = u∈K P(I(u) ∈ A u )F h9un point de vue physiqueD une imge peut être onsidérée omme une oservtion ruitée d9un phénomène sousEjentF n moyen e0e de pturer ette reltion physique est d9utiliser un dre yésien pour modéliser l9imgeF n modèle yésien d9une imge peut être érit ommeX P(I = y ∈ A) = x P(I = y ∈ A|X = x)P(X = x) @fFPA où P(I = y ∈ A) est l loi mrginle des oservtionsD P(I = y ∈ A|X = x) est l loi de l9oservtion et P(X = x) est l loi priori qui intègre l onnissne prélle sur le phénomène X sousEjentF in utilisnt le théoréme de fyesD nous pouvons nlyser le phénomene X à prtir d9oservtions y 1 , • • • , y n omme suitX P(X = x|I = y) ∝ P(I = y|X = x)P(X = x). @fFQA ve modèle de fyes est intéressnt d9un point de vue sttistiqueD r il fournit une lsse d9estimteurs nturelsX les estimteurs yesiens wrinD PHHUD qelmn et lFD PHIRF n exemple d9estimteur yesien est le ritère de wximum e osteriori @weA qui peut être érit omme suitX = x|I i = y i ) @fFRA qund toutes les oservtions y 1 , • • • , y n sont onditionnellement indépendntsF B.3.1.2 Processus ponctuels marqués pour l'analyse d'image ne desription détillée des proessus pontuels est donnée dns le ghpitre QF our l9instntD il su0t de noter que le proessus de point mrqué peut être utilisé pour modéliser des on(gurtions létoires de formes géométriques simplesF oit X : x = {x 1 , • • • , x n(x) }D où x i , i = 1, n(x) est un ojet géométrique plé u hsrd dns une imge et n(x) est le nomre de es d9ojetsF uisD dns un dre yésienD un modèle d9oservtion est donnéD pr exempleD prX P(I(u) = y u |X = x) = 1(u ∈ x)P(I(u) = y u |u ∈ x) + 1(u / ∈ x)P(I(u) = y u |u / ∈ x).

F

  Photogrammetric Engineering and Remote SensingD UIXITW!IUUD PHHSF @gited on pge VIFA FtF hveyD wFqF uttenD nd fF gheungF e omprison of detetion performne for severl trkEeforeEdetet lgorithmsF Proc. FUSIOND pges RWQ!SHHD PHHVF @gited on pge PTFA pF he ghumontD F hllongevilleD xF ghenourdD xF rerveD F opD et lF syX n open ioimge informtis pltform for extended reproduile reserhF Nature MethodsD W@UAXTWH!TWTD PHIPF @gited on pge IPQFA pF de ghumont et lF syX n open ioimge informtis pltform for exE tended reproduile reserhF Nature methodsD WXTWH!TWTD PHIPF v http: //icy.bioimageanalysis.orgF @gited on pges ID PD IPP nd IPTFA pF hell9equD gF fignmiD wF ghiniD qF visiniD hFeF olliD nd F trmondoF irthquke dmges rpid monitoring y stellite remote sensing dtX v9equilD epril TthD PHHW eventF JSTARSD R@RAXWQS!WRQD PHIIF @gited on pges II nd ISRFA F helworlD eF fouetD nd eF tsrF en dptive sequentil wonte grlo method for pproximte fysin omputtionF Stat. Comput.D PPXIHHW!IHPHD PHIIF @gited on pge RPFA eFF hempsterD xFwF virdD nd hFfF uinF wximum likelihood from inomplete dt vi the iw lgorithmF Journal of the royal statistical societyD QW@IAXI!QVD IWUUF @gited on pges RI nd ITVFA F heriheF sing gnny9s riteri to derive reursively implemented optiml edge detetorF IJCVD pges ITU!IVUD IWVUF @gited on pge TVFA

  wF yrtnerD F hesomesD nd tF eruiF e mrked point proess of retngles nd segments for utomti nlysis of higitl ilevtion wodelsF IEEE Transactions of Pattern Analysis and Machine IntelligenceD QHXIHS!IIWD PHHVF @gited on pges IQD TW nd ISSFA xF ytsuF e threshold seletion method from gry level histogrmsF IEEE Trans.Systems, Man and CyberneticsD WXTP!TTD IWUWF @gited on pge VHFA wF eF Stochastic models and methods for multi-object trackingF hh thesisD niversité ienes et ehnologiesD fordeuxD PHIIF @gited on pge QIFA FF heoF Parallel programming with MPIF worgn uufmnnD IWWTF @gited on pge UVFA gF pgeorgiou nd F oggioF e trinle system for ojet detetionF IJCVD QVX IS!QQD PHHHF @gited on pge PPFA pF pngelouF he onditionl intensity of generl point proesses nd n ppliE tion to line proessesF Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte GebieteD PV@QAXPHU!PPTD IWURF @gited on pge QTFA pF piD fFF oD wF foquelD nd fFxF oF qenerlized leled multiEfernoulli pproximtion of multiEojet densitiesF IEEE Trans. Signal ProcessingD TQ@PHAX SRVU!SRWUD PHISF @gited on pges PT nd QIFA wF elillo nd iFF rnokD editorsF Energy minimization methods in computer vision and pattern recognition: International Workshop EMMCVPR 1997F pringerD IWWUF @gited on pge RHFA 186 Bibliography eFqFeF ererD gF rinivsD eF roogsD qF frooksyD nd F ruF wultiEojet trking through simultneous long olusions nd splitEmerge onditionsF Proc. of CVPRD PHHTF @gited on pges PID PSD PV nd PWFA F érezF Modèles et algorithmes pour l'analyse probabiliste des imagesF rilittion à diriger des reherhes @rhAD niversité de ennes ID PHHQF @gited on pges IT nd ISWFA qF errinD F hesomesD nd tF eruiF e mrked point proess model for tree rown extrtion in plnttionsF Proc. ICIPD IXTTI!TTRD PHHSF @gited on pges IQD IUD IVD PRD ISSD ITH nd ITIFA wF irdiF fkground sutrtion tehniquesX reviewF Proc. ICSMCD pges QHWW!QIHRD PHHRF @gited on pge PSFA xFqF ltonovD sFxF wordvintsevD nd FF ozhnovF he possiility of using high resolution stellite imgery for detetion of mrine mmmlsF Bio BullD RHXIWU! PHSD PHIQF @gited on pges II nd ISQFA tFwF olndD editorF Understanding terrorism: groups, strategies and responsesF rentie rllD PHIHF @gited on pges II nd ISQFA tF ortillD F trelD wFtF inwrightD nd iFF imonelliF smge denoising using sle mixtures of qussins in the wvelet dominF IEEE Trans. PAMID IP@IIAX IQQV!IQSID PHHQF @gited on pges IR nd ISUFA F rveenD qF urthikD tF erivukrsiD F enrsnD nd F jeshwriF eognize irrft using templte mthing in high resolution stellite imgesF IJARSETD P@QAXSIV!SPID PHISF @gited on pges Q nd PQFA xF roiF Surveillance maritime par analyse d'images satellitaires optiques panchro-matiquesF hh thesisD niversité des entillesEquyneD PHIHF @gited on pges PQ nd WUFA tF rokj nd qF wedioniF ersistent trking for wide re eril surveiilneF Proc. CVPRD PHIRF @gited on pges PR nd PWFA F eilyD rF sdreesD nd wF hhF hetetion nd trking of lrge numer of trE gets in wide re surveillneF Proc. ECCVD veture xotes in gomputer iene @vxgAXIVT!IWWD PHIHF @gited on pges PI nd PUFA fF istiD F erulmplmD nd xF qordonF Beyond the Kalman lter: Particle lters for tracking applicationsF erteh rouse snFD PHHRF @gited on pge PTFA rF oins nd F wonroF e stohsti pproximtion methodF The Annals of Mathematical StatisticsD PP@QAXRHH!RHUD IWSIF @gited on pge RRFA gF F oert nd qF gsellF Monte Carlo Statistical Methods (Springer Texts in Statistics)F pringerEerlg xew orkD snFD PHHSF @gited on pges RSD UQ nd IHVFA qFyF oerts nd tFF osenthlF yptiml sling for vrious metropolisEhstings lgorithmsF Statistical ScienceD IT@RAXQSI!QTUD PHHIF @gited on pge IRQFA sF leemi nd wF hhF wultifrme mnyEmny point orrespondene for vehile trking in high density wide re eril videosF IJCVD IHR@PAXIWV!PIWD PHIQF @gited on pges PID PS nd PVFA wF hmitt nd tF wttioliF Morphologie mathématiqueF rnsvlorEressess des winesD PHIQF @gited on pge WSFA vF hoD F nD F viD nd F viuF prom heuristi optimiztion to ditionry lerningX review nd omprehensive omprison of imge denoising lgorithmsF IEEE Trans. CyberneticsD RR@RAXIHHI!IHIPD PHIRF @gited on pges IR nd ISUFA eFgF hstry nd FeF hozengerdtF eirorne video registrtion nd tr0E)ow prmeter estimtionF IEEE Trans. on Intelligent Transportation SystemsD T@RAX QWI!RHSD PHHSF @gited on pge PSFA rF henD F viD gF huD rF ghngD nd tF hngF woving ojet detetion in eril video sed on sptiotemporl slienyF Chinese Journal of AeronauticsD PT@SAX IPII!IPIUD PHIQF @gited on pge PWFA wF im nd wF ilrelwF oust utonomous visul detetion nd trking of moving trgets in e imgeryF Proc. of ICSPD PHIPF @gited on pges PID PT nd PWFA F ingh nd F lwrF i'ets of topogrphi orretions on wyhs sensor stellite imgery of mountins regionF Proc. of ICSCD pges RSS!RTHD PHIQF @gited on pges IP nd ISRFA eFFwF meuldersD hFwF ghuD F guhirD F gkderrD eF heghghnD nd wF hhF isul trkingX n experimentl surveyF IEEE Trans. PAMID PHIQF @gited on pge PVFA eF rivstvD wF willerD nd F qrennderF wultiple trget diretion of rrivl trkingF IEEE Trans. IPD RQ@SAXPVP!PVSD IWWSF @gited on pge SVFA eF rivstvD F qrennderD qF tensenD nd wF willerF tumpEdi'usion mrkov proesses on orthogonl groups for ojet pose estimtionF Journal on Statistical Planning and InferenceD IHQ@IGPAXIS!QUD PHHPF @gited on pge SVFA gF tu'er nd FiFsF qrimsonF edptive kground mixture models for relEtime trkingF Proc. of CVPRD pges PRT!PSPD IWWWF @gited on pge PSFA FF toiD F qregoriD nd tF wteuF imulted nneling nd ojet point proE essesX tools for nlysis of sptil ptternsF Stochastic Processes and Their Ap-plicationsD IISXIVTH!IVVPD PHHSF @gited on pge THFA Bibliography hF toyn nd rF toynF Fractals, Random Shapes and Point Fields: Methods of Geometrical StatisticsF tohn iley nd onsD IWWRF @gited on pges IQD UR nd ISSFA hF toynD F F uendllD nd tF wekeF Stochastic Geometry and its ApplicationsF tohn iley nd onsD IWVUF @gited on pges IQ nd ISSFA tFgF troeveD F uttsovD eF frrettD wF errezeD F vlovD wF rollndD nd FxF weierF rends in erti se extent from gwsSD gwsQ nd oservtionsF Geophysical Research LettersD QW@ITAD PHIPF @gited on pges II nd ISQFA F zeliskyF Computer vision: Algorithms and ApplicationsF pringerD PHIIF @gited on pge PIFA wF j nd eF gvllroF wultiEmer trkEeforeEdetetF Proc. ICSSCD PHHWF @gited on pge PTFA uF nk nd rF jiF ehile extrtion from eril imges using voting proess nd frme mthingF Proc. IVD PHHUF @gited on pge PTFA wF eutshF Moving object detection and segmentation for remote aerial video surveillanceF hh thesisD urlsruher snstitut fur ehnologieD PHISF @gited on pges PRD PT nd PVFA FF rinderD wFsF goleD uF rmmondD rFEF voidlD nd qFtF wihelsonF esoure nlyses for prllel nd distriuted oordintionF Concurrency and computation: practice and experienceD pges QHW!QRVD PHIQF @gited on pge UVFA F urmerD pF uurzD F einrtzD nd F tillF eirorne vehile detetion in dense urn res using roq fetures nd disprity mpsF JSTARSD T@TAXPQPU!PQQUD PHIQF @gited on pge PPFA F llmnF High-level vision: Object recognition and visual cognitionF ws ressD IWWTF @gited on pge PQFA wFxFwF vn vieshoutF tohsti nneling for nerestEneighor point proesses with pplition to ojet reognitionF Advances in Applied ProbabilityD PTXPVI! QHHD IWWRF @gited on pge THFA wFxFwF vn vieshoutF Markov Point Processes and Their ApplicationsF smperil gollege ressD PHHHF @gited on pge RIFA wFxFwF vn vieshout nd eFtF fddeleyF ixtrpolting nd interpolting sptil ptternsF sn eFfF vwson nd hFqFF henisonD editorsD Spatial Clustering Mod-ellingF gg ressGghpmn 8 rllD PHHPF @gited on pges IQD RI nd ISSFA F pnikF Statistical learning theoryF ileyD IWWVF @gited on pge PTFA F erdieF Modelisation de scènes urbaines à partir de donnèes aeriennesF hh thesisD niversité de xie E ophi entipolisD PHIQF @gited on pges QD SS nd SUFA F erdié nd pF vfrgeF i0ient wonte grlo smpler for deteting prmetri ojets in lrge senesF Proc. ECCVD USURXSQW!SSPD PHIPF @gited on pges SRD SSD STD USD UUD UWD IIWD IPHD IRV nd IUPFA F iol nd wF tonesF pid ojet detetion using oosted sde of simple feturesF Proc. CVPRD PHHIF @gited on pge PPFA F iol nd wF tonesF oust relEtime fe detetionF IJCVD SU@PAXIQU!ISRD PHHRF @gited on pge PPFA fFExF o nd FEuF wF he qussin wixture roility hensity pilterF IEEE Trans. Signal ProcessingD SR@IIAXRHWI!RIHRD PHHTF @gited on pge QIFA fFExF oD FF inghD nd eF houetF equentil wonte grlo implementtion of the rh (lter for multiEtrget trkingF Proc. Int. Conf. Information FusionD pges UWP!UWWD PHHQF @gited on pge QIFA fFExF oD FF inghD nd eF houetF equentil wonte grlo methods for multiE trget (ltering with rndom (nite setsF IEEE Trans. Aerospace and Electronic SystemsD RI@RAXIPPR!IPRSD PHHSF @gited on pge QIFAfFExF oD eF shD nd rFhF unF e qussin wixture rh (lter for nonliner jump wrkov modelsF IEEE Conf. on Decision and ControlD IXQITP!QITUD PHHTF @gited on pge QIFA fFExF oD fFEF oD nd hF hungF veled rndom (nite sets nd the fyes multiE trget trking (lterF IEEE Trans. Signal ProcessingD TP@PRAXTSSR!TSTUD PHIRF @gited on pge WIFA fFEF o nd fFExF oF veled rndom (nite sets nd multiEojet onjugte priorsF IEEE Trans. Signal ProcessingD TI@IQAXQRTH!QRUSD PHIQF @gited on pge WIFA fFEF oD fFExF oD nd eF gntoniF he grdinlity flned wultiEtrget wultiE fernoulli (lter nd its implementtionsF IEEE Trans. Signal ProcessingD SU@PAX RHW!RPQD PHHWF @gited on pge QIFA fFEF oD fFExF oD xFEF hmD nd hF uterF toint detetion nd estiltion of multiple ojets from imge oservtionsF IEEE Trans. Signal ProcessingD SV@IHAX SIPW!SIRID PHIHF @gited on pge TSFA F oigtD F uemperD F iedlingerD F uie)D uF holteD nd rF wehlF tellite imge nlysis for disster nd risisEmngement supportF IEEE. Trans. GRSD RS@TAXISPH!ISPVD PHHUF @gited on pges II nd ISRFA F uD fFExF oD nd FtF ivnsF e prtile mrginl wetropolisErstings multiE trget trkerF IEEE Trans. Signal ProcessingD TP@ISAXQWSQ!QWTRD PHIRF @gited on pge WIFA 190 Bibliography qFgFqF ei nd wFeF nnerF e wonte grlo implementtion of the iw lgorithm nd the poor mn9s dt ugmenttion lgorithmsF Journal of the American Statistical AssociationD VS@IRIAXTWW!UHRD IWWHF @gited on pge RQFA F ei nd vF oF i0ient histogrmEsed sliding windowF Proc. CVPRD PHIHF @gited on pge PPFA qF elh nd qF fishopF en introdution to the ulmn (lterF Proc. SIGGRAPHD pges IW!PRD PHHIF @gited on pge IPUFA gF uD F goD F vinD nd pF ngF egistrtionEsed moving vehile detetion for lowEltitude urn tr0 surveillneF Proc. WCICAD PHIHF @gited on pge PVFA FxF uD iD rF qongD nd FgF huF verning tive sis model for ojet detetion nd reognitionF IJCVD pges I!QVD PHHWF @gited on pge PVFA tF ioD rF ghengD rF whneyD nd pF rnF ehile detetion nd trking in wide (eldEofEview eril videoF Proc. CVPRD PHIHF @gited on pges PID PSD PT nd PWFA pF oD eF ekmenD nd wFtF wlkniF wultiple moving trget detetionD trking nd reognition from moving oserverF Proc. ICIAD PHHVF @gited on pge PWFA eF ilmzD yF tvedD nd wF hhF yjet trkingX e surveyF ACM Computing SurveysD QV@RAD PHHTF @gited on pge PVFA xF okoy nd eF swskiF yjet detetion sed on sprse representtion nd rough voting for optil remote sensing imgeryF IEEE. JSTARSD pge epted for pulition on PTGHIGPHISD PHISF @gited on pges QD PQ nd PRFA F u nd qF wedioniF sntegrted detetion nd trking for multiple moving ojets using dtEdriven wgwg dt ssoitionF Motion and Video computingD pges I!VD PHHVF @gited on pges PV nd PWFA F u nd qF wedioniF wultipleEtrget trking y sptioEtemporl wonte grlo wrkov hin dt ssoitionF IEEE Trans. PAMID QI@IPAXPIWT!PPIHD PHHWF @gited on pge IHSFA F uD qF wedioniD nd sF gohenF wultiple trget trking using sptioEtemporl wrkov hin wonte grlo dt ssoitionF Proc. of CVPRD pges I!VD PHHUF @gited on pge PWFA F ji nd F whlerF e prtileEsystems implementtion of the rh multitrget trking (lterF Signal Processing, Sensor Fusion and Target RecognitionD SHWTX PWI!PWWD PHHQF @gited on pge QIFA rF hngD hF ipfD nd F hngF wultiEimge lind delurring using oupled dptive sprse priorF Proc. CVPRD pges IHSI!IHSVD PHIQF @gited on pges IR nd ISUFA Bibliography 191 F hngD F unD uF puD gF ngD nd rF ngF yjet detetion in highE resolution remote sensing imges using rottion invrint prts sed modelF IEEE Geosci. Remote Sens. Lett.D II@IAXUR!UVD PHIRF @gited on pge PRFA F hengD qF houD F ngD F viuD et lF e novel vehile detetion method with high resolution highwy eril imgeF JSTARSD T@TAXPQQV!PQRQD PHIQF @gited on pge PTFA fF itovà nd tF plusserF smge registrtion methodsX surveyF Image and Vision ComputingD PIXWUU!IHHHD PHHQF @gited on pges IR nd ISUFA Géométrie stochastique pour la détection et le suivi d'objets multiples dans des séquences d'images haute résolution de télédétection

  Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.3 Approach and proposed methods . . . . . . . . . . . . . . . . 16

	Introduction
	Contents
	1.1

1.3.1 Probabilities in image / video analysis . . . . . . . . . . . . . 16 1.3.2 Application to object detection and tracking in videos . . . . 18 1.4 Thesis organization and contributions . . . . . . . . . . . . . 19 1.5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

  Object detection . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.1 Object detection in static images . . . . . . . . . . . . . . . . 22 2.1.2 Object detection in dynamic images . . . . . . . . . . . . . . 24 2.1.3 A note on person detection . . . . . . . . . . . . . . . . . . . 27 2.2 Object tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2.1 Data association based approaches . . . . . . . . . . . . . . . 28 2.2.2 Finite Set Statistics based approaches . . . . . . . . . . . . . 29

	Contents
	2.1

  otherwise @PFQA es the kground lerning n only e performed in the overlpping re of onE seutive frmesD it is importnt to note tht the more frmes re usedD the smller the kground model will eF fkground sutrtion n e used in ddition to di'erene imges to detet stopping ojets io et lFD PHIHF glustering of lol motion fetures is ompletely di'erent from di'erene imge omputtion or kground lerningF sf the ltter two re typil hoies for few uthors spei(lly trget people detetion in topEview high ltitude eril imgeryF he min reson is tht with top view mer @whih is the min se onsidered in this thesisA people n hrdly e deteted nd reognizedD s they not only pper very smll @round 5 -10 pixelsA ut n esily e onfused with shdowsF pigure PFQ shows visuliztion of multiple persons in eril nd stellite imgesF swshit et lF PHIH propose n pproh to person detetion in eril imges sed on their shdowsF roweverD shdow sed pprohes re useful only in the ontext of good lightning nd wether onditionsF e shdow silhouette normliztion is performed using metdt suh s timeD mer position nd sun positionF hdow fetures re omputed nd nlyzed to identify the presene of personF eily et lF PHIH produe initil detetions sed on grdientsD geometri onstrins nd ojetEshdow reltionships nd then omputes rr fetures for w lssi(tion with dil fsis puntion kernelF st is importnt to note tht people detetion in the ontext of eril imgery is omE pletely di'erent from people detetion in ground imgery nd most of the extensive reserh in the ltter se is not pplile for eril imgeryF

	2.2. Object tracking	Chapter 2. State of the art Chapter 2. State of the art	29
	e good survey on ojet trking methods is provided y ilmz et lF PHHTD while yh et lF PHHR propose relEtime lgorithm nd show tht this pproh exhiits oth the elements nd the rdinlity of the set re rndomF husD reliztion
	meulders et lF PHIQ gives more reent survey inluding omprehensive experE n inresed performne ompred to wr nd thep under extreme onditions of p is (niteD unordered set of elements distriuted ording to ommon
	imentl studyF suh s dense environments with lrge numer of trgetsD high flse lrm rtes nd distriutionF his pproh does not require ny dt ssoition omputtionsD
	he omplexity of the ojet trking methods depends on the omplexity of the low detetion proilitiesF his work hs een improved upon y u et lF PHHU sine it diretly propgtes the posterior intensity of the p of ojets in timeF
	dtF imple trking methods n e used in ses where only 10 -20 ojets nd u nd wedioni PHHVF u et lF PHHU expliitly use the sptioEtemporl e detiled desription of p nd pinite et ttistis @psA n e found in
	re present in the sene nd smll numer of split nd merge situtions our smoothness in motion nd pperne for their hhEwgwghe implementtionD qoutsis et lFD PHIPF he multiple ojet trking prolem n e posed s
	eutshD PHISF ytherwiseD one must employ more omplited trking methods while u nd wedioni PHHV provide n integrted detetion nd trking pproh fyesin (ltering prolem if the oservtions nd ojets re modeled s single
	pigure PFQX epperne of people in top view eril nd stellite imgesF tht hndle split nd merge situtionsD ojet olusion nd vrious motion modelsF for multiple moving ojetsF metEoservtion nd metEstteF
	sf point trking is desiredD extended trgets n e onverted to point representE por extended ojetsD dt ssoition n e solved y omputing the mount of he optiml multiple ojet fyes (lter n e gin divided into predition nd
	tions tones et lFD PHHSF e omintion of the position of the enter of the ojet overlp etween ounding oxes in onseutive frmes im nd ilrelwD PHIPF updte stepF uppose tht the multiple trget posterior density t time t -1 is 2.1.3 A note on person detection nd its veloity is ommonly used for point trking erer et lFD PHHTD leemi yther pprohes for solving dt ssoition n e performed sed on ojet reE p t-1|t-1 (X|Z 1:t-1 ) nd the umultive mesurements Z 1:t-1 = {Z 1 , • • • , Z t-1 } re
	nd hhD PHIQF ixtended trgets re usully desried y ounding oxes vi identi(tion using similrity mesures or grph mthingF srhim et lF PHIH use knownD then predition of the multiple trget posterior density p t|t-1 (X|Z 1:t-1
	et lFD PHHWD u et lFD PHIHD ellipses u et lFD PHHW or los srhim et lFD lo entroidD reD eentriity nd olor informtion to represent ojets nd then
	PHIHF gontour trking is less populr representtion for ojet trking in eril omputer similrities mesure for these lo fetures to solve the mesurementEtoE
	imgesD s the trgets re mnEmde ojets with rigid shpe tht n e usully trk ssoition prolemF o et lF PHHV evlute sptil nd olor similrityF
	represented y retngles @for vehilesA or ellipses @for otsAF io et lF PHIH use grph frmework to model olorD shpeD pperne nd spE
	en importnt hllenge in trking multiple extended ojets is the ssoition of til fetures nd ssoite detetions with existing trks through grph mthingF
	the detetions to existing trjetories ghll et lFD PHIIF he detetions n e rokj nd wedioni PHIR use the position of the detetion nd its pperne to
	split or merged whih yields neessity for inresed performne of the trking generte trklets whih they then omine to ojet trksF
	methodF en ojet is split if two or more detetions orrespond to the sme ojetD yne the dt ssoition prolem hs een resolvedD optiml (lter methods @suh
	while reverselyD ojets re merged if single detetion orresponds to more thn s the fyes (ltersA re usully pplied to updte n existing trk with n ssoE
	single ojetF ited detetionF he single trget fyes (lter onsists of two stepsD usully lled
	predition nd updteF uppose Z 1:t-1 = {z 1 , • • • , z t-1 } re the mesurements t
	time t -1 nd the posterior distriution is p t-1|t-1 (x|Z 1:t-1 )D then predition n 2.2.1 Data association based approaches e mde for time t s followsX
	hi'erent dt ssoition methods n e employed depending on the omplexity
	of the seneF he dt ssoition prolem for point trgets n e solved using p t|t-1 (x|Z 1:t-1 ) = f t|t-1 (x t |x t-1 )p t-1|t-1 (x t-1 |Z 1:t-1 )dx t-1 , @PFRA
	moving oE the nerest neighor erer et lFD PHHTD wultiple rypothesis rking leemi
	jetF xeverthelessD single motion re n ontin either just prt of the ojetD nd hhD PHIQD toint roilisti ht essoition pilter ung et lFD PHHSD u where f t|t-1 (x t |x t-1 ) is the wrkov stte trnsition model of the trget stteF yne
	multiple ojets or even no ojet t ll @eFgF noiseAF sndependent motion detetion n e regrded s redution of the serh speD sine only few smll regions need to e further proessedF eduing the serh spe is very importnt s it redues oth the omputtionl time needed to proess the video nd the numer et lFD PHHW or dtEdriven wgwgEsed dt ssoition u nd wedioni PHHVF mesurement z t is otined t time tD the posterior distriution n e updted 2.2 Object tracking en ojet trker is system tht genertes trjetories of the ojets over timeD y determining their positions in every frme of the videoF sdellyD n ojet trker he key priniple of wultiple rypothesis rking @wrA is tht di0ult dt ssoition deisions re deferred until more dt is reeivedF rypotheses re propE gted into the future relying on the ssumption tht susequent dt will resolve s followsX p t|t (x|Z 1:t ) = g t (z t |x t )p t|t-1 (x|Z 1:t-1 ) g t (z t |x t )p t|t-1 (x|Z 1:t-1 )dx , @PFSA
	of flse lrmsF yne the moving los hve een otinedD dditionl informtion suh s sizeD shpe nd eentriity n e used to lssify the lo s n ojetF eording to eutsh PHISD most ommonly pperne fetures suh s olorD shpe nd texture re lerned nd stored in model whih is then pplied y lssi(ers suh s w pnikD IWWVD ndom porests freimnD PHHI or oosting preund nd hpireD IWWUF wodel knowledge suh s lo motion gherghi nd heikhD PHIPD hs the following hrteristisX • st should detet ll ojets tht enter or move in the seneY • st should di'erentite etween di'erent ojets tht re present in the seneY • st should ssign n unique lel to eh ojet nd mintin it throughout the sequeneY the unertintyF where g t (z t |x) is the likelihood funtion t time tF everl implementtions of the he toint roilisti ht essoition pilter @thepA ssoites ll mesureE fyes (lter hve een devisedF he most populr re ulmn (ltering ulmn ments t time t with ll trksF he mesurementEtoEtrget ssoition proE IWTH nd prtile (ltering uitgw IWVUF erer et lF PHHT nd im nd ilities re omputed jointly over ll trgetsF fsed on the wrkov propertyD the ilrelw PHIP use ulmn (lters to updte the trks for point trgetsD while for thep omputes these ssoition proilities only for the ltest set of mesureE extended trgets vi et lF PHHW nd hen et lF PHIQ use men shift for ojet mentsF pinllyD the stte estimtion is done either seprtely for eh trgetD or in oupled mnner using stked stte vetorF representtions tht inlude motionD pperne nd shpeF
	lo shpe nd size heng et lFD PHIQ or shpe templte mthing nk nd jiD PHHU is used to determine the detetionsF pinllyD sliding window pprohes or prtEsed models n e used for deteting moving ojets strting from motion • st should hndle prtil or even totl olusion without hnging the lel of he dtEdriven wgwgEsed dt ssoition @hhEwgwgheA is ominE 2.2.2 Finite Set Statistics based approaches toril optimiztion pproh in whih the enumertion of trks is repled y the ojetY wgwg smpling methodsD suh s wetropolisErstings or qis smplingF he e di'erent frmework for multiple ojet trking is represented y ndom pinite
	losF hhEwgwghe ses its deision to form trk on urrent nd pssed oservtionsF • st should ope with senrios where the motion of the ojet hngesF ets @pAF e p n e desried s (niteD setEvlued rndom vrileD in whih

hetetEfeforeErk @hfA trking lgorithm viu et lFD PHHSD fugeu nd erezD PHHVD lol fetures lustering is hoie for the rkEfeforeEhetet @fhA trkE ing pproh hvey et lFD PHHVD j nd gvllroD PHHWF fh is the notion used for joint detetion nd trking of ojets nd ws initilly used for rdr dtD where the presene of noise is very strong isti et lFD PHHRF his is lso the reE son why fh is used in eril videos where the ojet motion ounts only for smll prt of the entire motion of the sene roseinnezhd et lFD PHIPD pi et lFD PHISF he ide ehind this method is tht moving ojet is expeted to produe severl nonEsprse moving lol fetures nd the motion res n e extrted y lustering these fetures sed on dditionl sptil nd motion onstrints vuo et lFD PHIPD im nd ilrelwD PHIPF his pproh hs some dvntges over difE ferene imges s it voids ghosting e'ets ut omes t higher omputtionl ostF sdentifying independent motion res does not neessrily imply the detetion of single moving ojetsF sdellyD one motion re orresponds to single ynly

  Marked pairwise interaction process) Let K be a compact set in R 2 such that 0 < ν(K) < ∞, where ν(K) is the Lebesque measure on K, and M = {1, • • • , I}, with I ∈ N, be the space of marks having the uniform probability measure ν M . A nite conguration of objects x = {x i

	IF he on(gurtion of ojetsD xD is unknownY
	PF he density funtion f (•) is known only up to normlizing onstntD cF
	purthermoreD the omputtion of this onstnt is nlytilly @nd usully
	lso numerillyA intrtleF
	he density of mrked point proess is hrterized y three types of prmeters
	ording to hesomes et lF PHIIX
	IF rmeters reltive to the proility density of the ojet mrkD or how the
	mrks re distriuted in the mrk speY
	PF rmeters reltive to the intertion (elds of the ojetsD or when do ojets
	intert with eh otherY
	QF rmeters reltive to the intertions etween the ojetsD or how do ojets
	intert within on(gurtionF
	Example 3.2.1 (
	3.2 Parameter estimation
	e generi model n e designed for solving generi prolemF sn order to dpt
	the generi model to spei( prolemD ertin prmeters of the model hve to
	e dptedF hese prmeters n e s stright forwrd s setting the mximum
	rdius of irle tht hs to e detetedD or s omplited s determining the
	weights of the di'erent terms within the density funtionF wo min di0ulties
	rise when deling with prmetri infereneX

  entered qussin distriution with vrine 1 n e used to simulte the disrete version of the frownin motion W u (τ i+1 ) -W

	60	Chapter 3. Using Point processes for object detection
	the density funtion must stisfy the vipshitzEontinuity ondition qemn nd
	rungD IWVTF
	3.3.3.3 Jump and diusion coordination
	he glol proess is ontrolled y relxtion prmeter lled tempertureF he
	ontinuous di'usion is interrupted y jumps t times t i whih re distriuted E
	ording to oisson distriutionF es the di'usion is disretized using time step
	δ i D the disrete witing timesD wD n e omputed sX
		w =	t i+1 -t i δ i	∼ p(w) =	τ w w!	exp -τ ,	@QFUSA
	where τ = E(w) represents the expeted witing time whih ontrols the frequeny
	of the jumps hesomes et lFD PHIIF
	3.3.4 Simulated annealing

u (τ i )F eording to hesomes et lF PHIID the jumpEdi'usion lgorithm is usully more e'etive thn the wgwg smplerF xeverthelessD the qis energy ssoited with he onvergene towrds n optimum for ll the smplers presented ove is gurE nteed y emedding the smpler in simulted nneling shemeF his is true when using the wximum vikelihood estimtorD lthough it is not mndtory if other estimtors re usedF imulted nneling ws developed y wetropolis et lF IWSQ nd proposes to reh the glol mximum of density h(•) y onstruting sequene of densities of the formX

  to drive the speed of the temperture dereseF his sheme proposes the use of plteu of length N F his plteu is prtiulrly useful to test the onvergene of the proessF pinllyD the temperture derese funtion does not hve to e (xedD it n e dptive s proposed y yrtner frmework of point proesses nd in prtiulr sptil nd sptioEtemporl mrked point proesses is suitle tool for extrting nd trking ojets in high resolution stellite imgesF qeometri properties n e esily represented into the frmework y mrks nd thusD the imge or video nlysis n e performed t ojet level whih is more roust to noise s opposed to pixelEsed pprohesF wthemtil tools for optimizing mrked point proesses models hve een suE essfully developed over the pst dede whih llow n e0ient simultion of suh modelsF es resultD dtEprllel implementtion of the twgwg smpler emE edded in simulted nneling sheme hs een developed whih not only hs good onvergene properties ut does this in reltively short mount of timeF he inresed )exiility in model designD the good onvergene properties of the twgwg smpler nd the reltively short omputtion time required for the optiE miztion proess to reh strong lol mximum mke this frmework prtiulrly ttrtive for ojet detetion nd trking in remotely sensed stellite dtF

	62	Chapter 3. Using Point processes for object detection
	3.4 Conclusions	Chapter 4
		Spatial marked point process
		model for boat extraction in
			harbors
	et lF PHHUF
	3.3.4.3 Adaptive decrease

he dptive derese funtion llows the temperture to ustom itself sed on the evolution of the energy of the systemF everl suh shemes hve een proposed in litertureF he interested reder n refer to yrtner et lFD PHHU for more detilsF he Contents 4.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.1.1 External energy term . . . . . . . . . . . . . . . . . . . . . . 65 4.1.2 Internal energy term . . . . . . . . . . . . . . . . . . . . . . . 67 4.1.3 Total energy term . . . . . . . . . . . . . . . . . . . . . . . . 71 4.2 Parameter estimation . . . . . . . . . . . . . . . . . . . . . . . 71 4.2.1 Determining the weights γ cnt and γ al . . . . . . . . . . . . . . 73 4.2.2 Determining the weight γ c . . . . . . . . . . . . . . . . . . . . 73 4.2.3 Determining the threshold d 0 . . . . . . . . . . . . . . . . . . 74 4.3 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.3.1 Perturbation kernels used for object detection . . . . . . . . . 75 4.3.2 Ecient implementation of RJMCMC -multiple cores . . . . 77 4.3.3 Water / Land discrimination . . . . . . . . . . . . . . . . . . 80 4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.4.1 Detection accuracy of the proposed model . . . . . . . . . . . 83 4.4.2 Computational eciency . . . . . . . . . . . . . . . . . . . . . 86 4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 es presented in hpter ID ot extrtion in hrors is di0ult prolem due to the prtiulr distriution of the

  qrphil roessing nit @qA prllel implementtion of the twgwg smpler proposed y erdié nd vfrge PHIP is sed on the underlying ssumpE tion tht ojets do not ross the oundry of ell more thn ertin smll vlue εF his ssumption enles the use of dtEprllel proessing model wttson et lFD PHHSF sn other wordsD there is no need for individul proessors to ommuE nite nd shre dt etween eh otherD nor to ess shred memoryF heir ssumption is generlly true for smll ojetsF roweverD if the ojets re lrgeD this ssumption is no longer vlidF e lrge ojet n e ontined in two neighoring ellsD s shown in pigure RFV @leftAF smposing dtEprllel proessing in eh ell independent from the on(gurtions in the neighoring ells results in two possile types of detetion errorsX IF he ojet is split t ell oundry nd thusD two smller ojets re detetedD s shown in pigure RFV @middleAY PF illipses re llowed to ross the ell oundry whih would led to severl detetions of the sme ojetD s shown in pigure RFV @rightAF 4.3.2.1 Challenges in exploiting parallelism everl hllenges hve to e onsidered whih rise with the inrese in proessing elements nd the need to oordinte ess to shred resouresF rllel progrmE ming @iFeF providing orret nd e0ient progrm tht tkes full dvntge of the Chapter 4. Spatial marked point process model for boat extraction in harbors underlying prllel rhitetureA is onsiderly more hllenging thn sequentil progrmming due to the dded omplexity of oordintion etween the proessing elements posterD IWWSD rinder et lFD PHIQF he following spets re key issues tht hve to e onsidered when developing prllel implementtion felikov et lFD PHIQX • CoordinationF goordintion tkles the ess to shred resouresD prlE lelism nd interEproess ommunitionF iither the omputtion or the dt @or othA hve to e prtitioned to filitte prllel exeutionY • Performance portabilityF he performne portility refers to the ility of softwre to mintin high performne ross di'erent prllel rhiteturesY • ProductivityF righElevel progrmming models generlly o'er high produE tivity nd )exiilityY • ScalabilityF lility is the ility to inresingly use ville resoures suh s memory size or the numer of proessing elements to solve lrger prolemsF hen devising prllel implementtionD the progrmmer must tke ll these sE pets into ountF righElevel prllel progrmming models o'er n strtion of the omputer system rhiteture wttson et lFD PHHSF he most ommon highElevel prllel progrmming models reX • Shared memory model is multiEthreded model whose prtil impleE menttion is ypenwF ypenw ghpmn et lFD PHHUD ypenwD PHIQ enE les highly strutured use of thredsF wore preiselyD the swith etween sequentil nd prllel regions is performed through forkGjoin model enE drewsD IWWWF he ontrol thred uses fork ommnd to split into severl independent thredsF efter the threds (nish their exeutionD they join to resume the sequentil exeutionF he prllel region llows the replition of single tsk ross set of thredsY • Message passing model is spei(lly designed to enle ommunition etween proesses y interhnging messgesF his is ommon lterntive for distriuted systemsD where shred vriles nnot e used for ommunitionF he wessge ssing snterfe @wsA is the most widely used spei(tion for messge pssing opertions heoD IWWTD qropp et lFD IWWWF endGeeive opertions re enoded to filitte interEproess ommunitionY

RFPSA

pigure RFVX veftX e lrge ot split in two prts due to the spe prtitioningY widdleX he ojet is split t ell oundry leding to the detetion of two smller otsY ightX he ot is deteted twie if ellipses re llowed to ross the ell oundryF 4.3.2 Ecient implementation of RJMCMC -multiple cores he

  PHHS puts forwrd n nlysis of the shdows tht pper in high resolution optil imges of urn resF re identi(es one min feture tht n e used to distinguish etween shdows nd wterX intrEre vrineF es hre PHHS points outD the vrine of the rdiometri vlues within the shdow res tends to e lrger when ompred with the vrine within wter resD due to the underlying strutures on whih the shdow is stF his feture is used in onjuntion with the size feture to onstrut n lgorithm @see elgorithm VA to identify the wter reF he lgorithm is stright forwrdX fter pplying threshold to the initil imgeD the identi(edF por eh onneted omponentD its size nd the vrine of the rdiometri vlues inside the omponent re omputedF pinllyD only those omponents tht exhiit lrge size nd smll vrine re seleted s wter resF le RFI shows the sttistis for three shdow res ompred to wter reF elthough the men rdiometri vlue is similr for oth lss typesD ler di'erene n e oserved in terms of size nd rdiometri vrineF pinllyD morphologil diltion is used to extend the wter re to inlude ots tht re nhored ner the lndF he resulting wter msk Ξ is depited in pigure RFW @ottomAF

	Algorithm 8 elgorithm for reting wter msk
	Input: snput imgeY
	Output: finry msk displying the wter re in the input imgeF
	IF hreshold the input imge using the threshold otined y imodl histogrm
	splittingY
	PF sdentify the onneted omponents in the threshold imgeY
	QF por eh onneted omponentD ompute its size nd the intrEomponent
	vrine of the rdiometri vluesY
	RF glssify the onneted omponents into wter nd shdows sed on the size
	nd vrine feturesF
	4.3.3.1 Water mask computation
	gomponent type	ize	wen	rine
	hdow 1	8641	12.8962	1.03389
	hdow 2	8211	13.0898	1.22846
	hdow 3	9986	13.0068	1.00696
	ter	969675	13.2784	0.377285
	le RFIX gomprison of hrteristis of wter nd shdow omponents
	onneted omponents re

hre

  Water mask integration into the RJMCMC sampler yne the wter msk hs een retedD we need to integrte it into the twgwg smplerF he wter msk Ξ represents the re where ojets re expeted to pperF e ssume tht no ojets exist outside this reF e integrte the wter msk y modifying the underlying oisson intensity ν(•) suh tht for n ojet uX

	4.4. Results				83
	4.3.3.2 ν(u) =	ε if u ∈ Ξ, 0 otherwise		@RFPUA
	reneD ojets will e reted only in the preEdetermined wter reF
	4.4 Results
					sene
	hve the sme orienttionF
		pigFRFIH	pigFRFIH	pigFRFIH	pigFRFIH
		@topA	@seond rowA	@third rowA	@ottomA
	xumer of ots @qA	518	178	572	131
	xumer of deteted ots	501	169	427	96
	reision	97.6%	94.6%	97.6%	87.5%
	ell	88.9%	87.9%	84.2%	80.0%
	eury	85.5%	81.7%	UWFR7	65.4%

sn the followingD detiled nlysis is presented of the results otined on two di'erent types of stellite imgesX

• pot 5 imges with 2.5m ground smpling resolution provided y gxiY

• leides imges with ground smpling resolution of 0.5m provided y eirus h8F he omputtionl e0ieny of the proposed method is lso disussed elowF 4.4.1 Detection accuracy of the proposed model sn the pigure RFIH the results otined on pot 5 imges over the prenh ost re depitedF he (rst imge is 385 × 275 pixels in size nd ll the ots in the

  Spatial marked point process model for boat extraction in harborshe seond imge is 326 × 226 pixels in size nd ontins ots tht hve two di'erE ent orienttionsF he third nd forth imges re 501 × 451D respetively 410 × 410 pixels in size nd the orienttion of the ots hs lrge vrine within the senesF he quntittive results re summrized in le RFPF he ground truth inforE mtionD the numer of ots deteted nd the detetion errors for eh imge re shownF he detetion results re very good nd the lgorithm orretly detets ots of di'erent sizesF his is minly due to the orienttion onstrint within the internl termF his onstrint fvors ots tht re oriented perpendiulr to the rough linesF he detetion results on two loseEups of leides imge of hror re in welE ourneD eustrli re illustrted in pigure RFIIF ine the ots re well seprted from eh otherD the detetion ury is very goodF le RFR shows the quntitE tive results for these two loseEupsF he imge on whih these loseEups were tken is 1620 × 1450 pixels in sizeF he loseEups re used for etter visul ssessmentF

	86 Chapter 4. 5 g9s	gomputtion times for pigure RFIH @topA	gomputtion times for pigure RFIH @seond rowA
				xo wter msk	ter msk	xo wter msk	ter msk
	1			2h 05min	2h 05min	1h 40min	1h 40min
	2			1h 17min	1h 17min	58h 12se	58h 12se
	8		25min 25se 25min 25se 14min 46se 14min 46se
	24	18min 58se 18min 18se 10min 04se 10min 04se
	5 g9s	pigFRFII @topA gomputtion times for pigure RFIH @third rowA	gomputtion times for pigFRFII @ottomA pigure RFIH @ottomA
	xumer of ots @qA xo wter msk xumer of deteted ots 1 2h 51min	32 ter msk 31 2h 12min	xo wter msk 1h 51min	42 ter msk 39 1h 27min
	2	reision	1h 31min	100% 1h 17min	1h 02min	100% 49min 39se
	8	ell	96.8% 29min 51se 27min 09se 10min 56se 6min 34se 92.8%
	eury 24 16min 11se 17min 34se 7min 47se 99.7%	99.2% 5min 13se
	le RFRX untittive nlysis of the proposed ot extrtion lgorithm for
	leides imgesF hue to the very high resolution of these imgesD the ojets no
	longer pper to e tngentF husD the overll ury is very goodF

  this hpterD new mrked point proess model hs een presented tht drws on stohsti geometry to extrt ots in hrorsF his prolem is hrd due to the prtiulr distriution of the otsF wost of the ots re very lose to eh other or even tngentD whih mkes individul detetion di0ultF purthermoreD ots n hve di'erent sizes or orienttionsD s well s drk spots whih mkes templte mthing tehniques ine0ientF he model proposed in this hpter hs signi(nt dvntgesX

		gomputtion times for pigure
	5 g9s	RFW @A
		xo wter msk	ter msk
	1	2h 4min 5se	1h 38min 57se
	2	1h 45min 41se	1h 20min 23se
	8	49min 23se	44min 7se
	24	12min 54se	9min 41se
	le RFSX untittive nlysis of the omputtionl e0ieny for ot extrtion
	on leides imgesF
	4.5 Conclusions

sn

  trgetF he link etween the externl energy term nd the level of flse lrms is not oviousF husD it is more di0ult to on(gure the prmeters of the model in order to reh desired level of detetionsGflse lrmsF xeverthelessD the imporE tne of the ontrst distne mesure eomes pprent when the segmenttion results re onsideredF his model o'ers high ontrol power over the (tness of on(gurtion wFrFtF the trget on(gurtionF he proposed method hs een pplied to lrge numer nd vrious types of dt sets within the sienti( (elds of remote sensing nd )uoresent mirosopyX enhE mrk syntheti iologil dt sets hve een extensively used to testD vlidte nd ompre the modelsF el iologil sequenes hve een used to show tht the models re suited for rel pplitionsF pinllyD results on high resolution stellite imge sequenes with high nd low temporl resolution hve shown the dvntges of the proposed methodF he high detetion nd trking rtes show tht sptioEtemporl mrked point proE ess models re well suited for relElife pplitions nd outperform urrent stte of the rt methods in these two dominsF Thesis contributions . . . . . . . . . . . . . . . . . . . . . . . . 147 6.2 Advantages of the proposed methods . . . . . . . . . . . . . 148 6.3 Drawbacks of the proposed methods . . . . . . . . . . . . . . 149 6.4 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 6.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . 150 6.1 Thesis contributions huring this hFhF thesis we studied how stohsti geometry n e e0iently emE ployed for the nlysis of high resolution optil imges nd videos tken y remote stellite sensorF he im of this work ws to investigte the possiility of using mrked point proesses to detet nd trk multiple moving ojetsD where the trgets pper smll in the imge with usul size of 10 E 100 pixels per ojetF he min ontriutions of this thesis reX IF A novel spatio-temporal marked point process model for the detection and tracking of moving objectsF e hve developed n intuitive model sed on solid theoretil kground whih inorportes onstrints on the sptil nd temporl distriution of the ojetsF he temporl inE formtion hs llowed us to restrit the serh for moving ojets to res tht exhiit signi(nt hnges in onseutive frmesF e hve pplied the model to detet nd trk vrious types of ojets inluding rsD ots nd pedestrins in stellite videos s well s vesiles in mirosopy imgesF hese pplitions hve enled the use of simple yet su0iently generl onstnt veloity motion modelF e hve used the otined trjetories to infer higher level informtion suh s the tr0 density estimtion @in urn resAY PF Improved spatial marked point process model for boat extraction in harborsF fot ounting in hrors is very di0ult tsk due to the prtiulr distriution of the ojetsF sn hrors ots re nhored very lose to eh otherF gonsequentlyD two or more neighoring ots re usully deteted s one lrge ojetF fen rdj et lF PHIH proposed model for this extrtion prolem when ll ojets hve the sme orienttionF e uild on the model presented y fen rdj et lF PHIH nd develop more generl nd e0ient model for ounting ots in hrors regrdless of their orienttionF sn prtiulrD we disrd the onstrints whih mke the model of fen rdj et lF PHIH too restritive nd introdue simple heuristis to improve the qulity of the resultY QF Automatic parameter estimationF he performne of mrked point proE ess models for spei( pplition depends on well suited nd e0ient hoie of prmetersF he models presented in this work re liner ominE tion of severl weighted energy termsF hese weights re generlly hrd to set mnully s they lk diret physil interprettionF e hve developed n utomti prmeter estimtion tehnique sed on liner progrmming tht n e e0iently used to set the prmeters of suh modelsY RF Integrated RJMCMC sampler with Kalman-like movesF twgwg dynmis re used to simulte the models presented in this thesisF twgwg is n itertive smpler tht uses set of perturtion kernels to simulte wrkov hin tht onverges to the desired distriutionF his smpler hs very good onvergene properties whih is why it is of gret interest to the sienti( ommunityF xeverthelessD these properties ome t high ompuEttionl ostF e hve proposed novel hyrid smpler y integrting the twgwg smpler with ulmnElike movesF he ulmnElike moves re used to diret the pth tken y the twgwg smpler to onverge more rpidly towrds the desired distriutionY trjetoryD their sizeD shpe nd orienttionF he integrted energy term oupled with the th optimiztion proess enles the detetion of wekly ontrsted ojets or temporrily oluded ojets while mintining onsistent trjetoriesF his pproh is utomti nd permits to struture the extrtion of trjetories sed on the intertions etween the ojetsF gomplex ojet intertions n e e0iently inorported into the modelsF en importnt dvntge of this method is its roustness to the type nd qulity of the dt usedF sn prtiulrD prmeter trining is performed only one for eh dt setF he model n then e suessfully pplied to eh sequene in dt setF e lso experiment with using the sme prmeters ross multiple similr dt sets nd show tht the qulity of the result is not signi(ntly 'etedF elthough the dvntges of suh method re solidD severl drwks hve een identi(ed during this reserh workF 6.3 Drawbacks of the proposed methods pirst nd foremostD the omputtionl e0ieny remins prolemti spetF he mthemtil frmework of mrked point proesses in whih this work is developed omes with n intrinsi omputtionl urdenF he use of e0ient heuristis we proposed in this work to redue the serh spe nd to speed up the optimiztion proess hs signi(ntly redued the omputtion timeF xeverthelessD in n er of relEtime proessingD these methods fil to keep up the peF sn the urrent settings s presented in this thesisD relEtime performne n e otined only for lowE frequeny videos with smll imges nd limited numer of ojetsF gonsequentlyD huge improvements hve to e mde in this regrd to render mrked point proess models suitle for multiple ojet trking in the futureF enother drwk of the proposed models is owed to the wek 9right ellipse9 trE get modelF yn the one hndD this trget model is simple enough to e e0iently inorported into the point proess modelF woreoverD for the pplitions presented in this thesis suh simple trget model su0es to otin highEqulity resultsF yn the other hndD the use of suh simple trget model proves to e limiting in terms of pplitionsF gonsequentlyD more elorte trget model tht n integrte pperne informtion would e desiredF 6.4 Perspectives sn this study we modeled the trking prolem t the level of ojetsF he interE tions etween these ojets re desried oth in the sptil nd temporl dominF emnti informtion out ojet trjetories is then retrieved y grouping the ojets sed on their lelsF en interesting pproh would e to design hierrE hil model tht integrtes oth lowElevel onstrints etween individul ojets nd highElevel onstrints etween trjetoriesF he geometri mrk of point within the point proess depends on the ppliE tion onsideredF imple shpes suh s retngles or ellipses re more desirle to hrterize the geometry of ojets s they re desried y smll numer of prmetersF sn prtiulrD we hose to use only ellipses throughout this work euse they (t most of the ojets of interest @eFgF rsD otsD vesilesD etFAF xeverthelessD n extension towrds the use of multiple shpes would e desiredF vE frge et lF PHIH propose multiEmrked proess to extrt ojets with di'erent shpes within n imgeF en extension to video dt n e envisioned to distinguish etween vrious ojet lssesF roweverD re must e tken to preserve the lss of n ojet throughout its trjetoryF e hve designed our models to extrt individul trjetories of moving ojetsF roweverD in some pplitions suh s tr0 monitoringD informtion out the tr0 density n tke preedene over individul trksF he notion of tr0 here is not restrited to vehilesD s it n lso inlude pedestrin nd mritime tr0 for stellite dt or ell memrne tr0 in mirosopy imgesF e model to estimte the density of ojets throughout the video rther thn individul trjetories n e of interest in suh pplitionsF pinllyD we hve proposed severl wys to improve the performne of the twE gwg smplerF iven soD the optimiztion proess should e further improved to mke suh models ompetitive with existent stte of the rt trking lgorithmsF 6.5 Concluding remarks his mnusript presents only prt of the work undertken during the lst three yersF e hve hd the opportunity to explore di'erent pprohes nd theories thn those desried hereD to experiment with lrge mount of dt nd to onE sistently reover from dedEends nd filed ttemptsF es suhD the lst yers hve een very instrutiveF his study hs llowed us to evlute the potentil of mrked point proess modE els for trking purposesF e hve shown tht suh models re strong ompetitors ginst stte of the rt methods with respet to the qulity of the resultsF etD onsistent improvements hve to e done to the optimiztion proedure in order to mke mrked point proess models leding hoie in the futureF eltogetherD the vst reserh work on ojet detetion nd trking in stellite s well s mirosopy imgery hs mjor impt on our dily livesF prom inresed seurity levels nd wildlife monitoring to suEellulr tivity nlysisD ojet deE tetion nd trking methods in these domins re one of the uilding loks of deeper understnding of our world nd the ptterns tht govern itF Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 B.2 Les dés . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 B.3 Méthodes et approches proposées . . . . . . . . . . . . . . . 159 B.3.1 Probabilités dans l'analyse des images / du vidéo . . . . . . . 159 B.3.2 Application à la détection et suivi des objets dans des vidéos 161 B.4 Organisation de thèse et contributions . . . . . . . . . . . . . 162 B.5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

	146 6.3. Drawbacks of the proposed methods Chapter 5. From object detection to object tracking using a spatio-temporal marked point process model 149 B.2. Les dés 155 B.2. Les dés 157
	Chapter 6 Conclusions and perspectives stellite dépend des pplitions spéi(ques pour lesquelles e stellite été onE struitF ves données stellitires ont déj été utilisées ve suès pour des opértions v9nlyse d9imges stellitires n9est ps le seul domine où les opérteurs humins sont surhrgés sous l quntité de données disponilesF v9imgerie mirosopique et de tritementD de l9quisition d9imges à l9nlyse de sèneF elon le moment et le lieu de l9ppritionD ils peuvent être lssées omme suitX générées pr de gros véhiules tels que les mions et les utous ou pr de grnds immeulesF r onséquentD ertins ojetifs ou iles potenE suh s their Introduction de reherhe et suvetge vukowski nd ghronneuD PHHHD ting nd hnpingD PHIID d9ide ux tstrophes nturelles oigt et lFD PHHUD hell9equ et lFD PHIID l qulité de l vidéo mirosopique ont onnu une forte ugmenttion es dernières nnéesF v9nlyse des données de mirosopie est désormis monnie ournte dns tielles pourrient être mnqués ou des omres pourrient être identi(ées IF Acquisition d'image / de vidéo omme des ilesY
	fhngle nd hurhD PHIR ou de surveillne de l9environnement gett et lFD PHIID ingh nd lwrD PHIQF ves progrès tehnologiques dns l9industrie sptile des domines tels que l médeine ou l reherhe iologiqueF v hute vitesse • Le bruit du capteur représente un ért pr rpport ux vleurs rE • Les mouvements indépendant du capteur et l'objet joute des d9quisition des mérs permet l9oservtion en temps réel de proessus elluE diométriques optimles des pixels de l9imgeF elon le type de pteur di0ultés supplémentires pour l détetion d9ojetsF v9enregistrement
	permettent désormis d9otenir des imges euoup plus préises et (les qui ont lires et suEellulires dynmiquesF in tnt que telleD ette pprohe peut générer utiliséD le ruit peut être modélisée soit omme dditifD multiplitif de l9imge et l déformtion itovà nd plusserD PHHQ sont utilisés pour
	he ttistil model is sed on sttistil hypotheses testing to determine whether n ojet is present t given lotionF hrough this pproh we n on(gure the prmeters of the model to reh n pplitionEdependent level of flse lrmsF xeverthelessD this model does not ontrol how n ellipse (ts the trget ojet whih leds to poorer segmenttion resultsF le potentiel de révolutionner les prtiques de surveillne u ours du PIéme sièleF de vstes quntités de données qui doivent être stokéesD tritées et nlyséesF hes @tveluresA ou impulsif @poivre et selA qonzlez nd oodsD PHHVY ompenser le mouvement de l mérF insuiteD les ojets en mouvement Contents 6.1 SF Ecient parallel implementation of the RJMCMC samplerF e suE essful ttempt to inrese the omputtionl e0ieny of the smpler ws mde y erdié nd vfrge PHIPF erdié nd vfrge PHIP propose prE llel implementtion of the smpler on q nd otin gret improvements in terms of omputtion time over the sequentil versionF hey use hierrhil in orretly extrting ojets t the ell oundryF e hve proposed new multiple ore implementtion of the twgwg tht keeps shred memory etween proessors so tht the perturtions in ell re performed y tking into ount its neighorhoodF his reserh work shows tht mrked point proess models n e used for trking purposesF e qulittive nlysis of the results revels high detetion nd trking performnes for ll the videos presented in this thesisF he dvntges of this pproh re disussed in the following setionF 6.2 Advantages of the proposed methods oursF wis qund es méthodes éhouentD l surveillne érienne et sptile peut ider à déteter les tivités riminelles vnt ou pendnt leur exéutionF ves êtres humins ne sont ependnt ps les seuls à pouvoir éné(ier de l surveilE lne érienne ou stellitireF vors de l derniére déennieD des entines d9espées d9nimux sont devenus des espèes menées pr l hsse exessive ou le réhufE fement plnétireF r exmpleD le réhu'ement de l plnéte onduit à une perte élérée de l nquise rtique u ours des derniéres nnées gomiso et lFD PHHVD troeve et lFD PHIP e qui un fort impt sur l fune rtiqueF v surveillne érienne peut ider à évluer les tendnes de l9évolution sttistique des espées d l9ertique pretwell et lFD PHIPD ltonov et lFD PHIQ sns se préouper u sujet des méthodes d9étude intrusives ou de l séurité humineF ves stellites peuvent être équipés d9une vriété de pteurs tels que des pteurs d9imges prises à intervlles de temps reltivement ourtsF v fréquene temporelle des vidéos peut être omprise entre environ 1 -2rz et 30rz ou plusD en fontion du type de pteur utiliséF r exmpleD un seul stellite en orite sse @omme kytE1 pr exempleA peut produire des imges de résoultion sousEmètrique et pturer des vidéos hute résolution jusqu9à 90 seonds à 30 trmes pr seondeF gependntD un grnd nomre de stellites en orite sse sont néessires pour ouvrir une zone terrestre donnée pendnt une longue période de tempsF i telle est l9intentionD un stellite géostrtionnire pourrit fournir une lterntive à une orite sse qui sont hélioEsynhronesA et peuvent ouvrir une plus grnde surfe u solF r onséquentD un tel stellite produirit de très grndes imges à une tels sY B.2 Les dés iple très fréquentée ou des teux dns les ports sont des exemples de hronesD e qui signi(e qu9ils tournent ve l erre @pr opposition ux stellites en omme une grnde ileF v irultion des véhiules sur une route prinE onstelltion de stellites en orite sseF ves stellites géosttionnires sont géosynE qui entrîne générlement que deux petits ojets voisins soient détetés mrqués ve un on ompromis entre l performne de suivi et d9exéutionF térisées pr un grnd nomre de iles situées près les uns des utresD une nouvelle pprohe de suivi d9ojets multiples fondée sur les proessus pontuels des ojets est très limitéeF r exmpleD des sènes urines sont rE odes existntes pr rpport à l préision de l détetion des ojets et de proposer puisque l quntité d9informtion disponile sur l9pprene et l forme et des pprohes nouvelles sont introduits (n d9dpter et d9méliorer les méthE e qui rend l détetion et l reonnissne prtiulièrement di0iles sousEjent de ette thèse est sée sur l littérture existnteD mis plusieurs idées est générlement de quelques pixels @iFeFvrint entre 10 et 100 pixelsA vent produire des résultts ompétitifs pour es tâhesF ve fondement théoretique d9éhntillonnge u sol des pteurs onsidérésF v tille des ojets d9imgerie optiqueF v9ojetif est d9exminer si les proessus pontuels mrqués peuE • La petite taille de l'objet est une onséquene de l résolution sptioEtemporelles mrqués pour l détetion et l suivi d9ojets dns di'érents types form independent perturtions in these ellsF gonsequentlyD di0ulties rise vnduD PHII ne sont ps su0sntes pour prévenir ou déteter des ttques en sl est importnt de noter que nous utilisons le terme vidéo pour une séquene gette thèse porte sur l9utilistion de modèles de proessus pontuels sptiux et QF Analyse d'image / de vidéo prtitioning sheme to divide the imge spe into independent ells nd perE Contents v dernière déennie été une vitrine pour l guerre symétrique ve des démonE strtions d9ttques hutement orgnisées et des méthodes de ommunition et de plni(tion vnées uydd nd lterD PHHTD olndD PHIHF in onséqueneD l séurité ivileD militire et éonomique est menée pr des tenttives d9ssssint ien doumentésD des prises d9otges et des ttques terroristesF hes méthodes popE ulires telles que l9nlyse des médis soiux puhsD PHHW ou l9éoute életronique réelF gette thèse porte sur l9nlyse d9imges sttiques et dynmiques provennt priniE plement de pteurs optiques emrqués à très hute résolutionF in prtiulierD nous tritons à deux prolèmes prinipuxX l détetion d9ojets dns des imges sttiques et l détetion et le suivi d9ojets en mouvement dns les vidéosF v déE tetion d9ojets etD respetivementD le suivi sont deux étpes intermédiires pour l ompréhension utomtique d9une sèneF ges motifs omplexes d9informtion de hut niveu peuvent ensuite être utilisées pour modéliser et déteter des omporteE ments suspets et des vleurs errntes peuvent être inditives de omportements riminelsF hes méthodes fondées sur des imges et des vidéos fournissent des inforE mtions préieuses omme de nomreuses propriétés sur les ojets détetésD omme pr exmple l positionD l tilleD l9pprne ou l forme d9un ojet peuvent être diretement dérivés à prtir de es donnéesF fonde et des tritements médiux plus déqutsF ves motifs géométriques omplexes néessitent souvent l9nlyse sttistiqueF hes outils sttistiques dptés et des modèles mthémtiques ppropriés sont néesE sires pour nlyser es donnéesF v géometrie stohstique est l9un des domines de l reherhe mthémtique qui vise à fournir de tels modèles et méthodesF v théorie moderne de l géométrie stohstiqueD mise u point pr hFqF uendllD uF somes et lFD PHIIF données xetrvli nd rskellD IWWSD entoniniD PHHQY et lFD PHHSD voste et lFD PHHSD hesmps et lFD PHHVD yrtner et lFD PHHVD heE omme des los qui peuvent dégrder onsidérlement l qulité des d9ojets dns des imges stellitires très hute résolution yrtnerD PHHRD errin • Les artefacts de compression-décompression sont des strutures pontuels mrqués ont déj été ppliquées ve suès à des prolèmes de détetion imges mnquntes qonzlezD PHIQY fddeleyD PHHPF ves proessus pontuels sptiux et en prtiulier les proessus perturée qui peut onduire à des rtefts puissnts ou même à des tions ompléxes toyn et lFD IWVUD toyn nd toynD IWWRD vn vieshout nd • Des artefacts lors de ette étpe peuvent être usées pr une onnexion urikererg et FiF wilesD exmine des motifs géométriques létoires de distriuE • Le contraste faible pprît en rison des onditions environnemenE tlesF ves onditions météorologiques spéi(ques telles que le rouillrd ou les nuges sont des situtions ourntes qui onduisent à un onstrste file dns les mérs optiquesY PF Transfer d'image / de vidéo B.1 B.1 Motivation uel que soit le type de pteurs montés sur le stelliteD l9nlyse des données E quises est un trvil di0ile et fstidieux pour les opérteurs humins rtérisé pr un niveu élevé de ftigue et d9ennui qriD PHHU en rison de l grnde quntité d9informtions disponiles dns es donnéesF hes lgorithmes ppropriés pour le tritement de données et l reherhe d9informtion utomtique et semiE utomtique dns des imges peuvent ider l9opérteur huminF xénmoinsD il est di0ile pour l pluprt des pplitions de grntir un tux d9erreur file et une grnde on(ne dns un lgorithmeD tout en o'rnt une performne qusiEtemps méthodes utomtiques et semiEutomtiques pour l réupertion de l9informtion sont d9une grnde importne pour l9opérteur humin dns e domineF r onE peuvent être détetés pr leur mouvement reltif pr rpport à l9rrièreE • Les artefacts sont des strutures rti(ielles qui sont ontenues dns pln (xeY l9imge et représentent une perturtion du signlF hes exemples d9ojets séquentD nous testons églement les modèles développés dns ette thèse sur des dns ette étpe omprennent l sturtion du pteur ou des prolèmes • Les exigences en temps de calcul posent un grnd dé( pour les lgoE séquenes d9imges iologiques synthétiques et réellesF xous sommes prtiulièreE de onversion e G x @nlogique G numériqueA qonzlezD PHIQY rithmes de suiviF hns l pluprt des pplitionsD un ompromis entre l ment intéressés pr l détetion et le suivi des strutures suEellulires ppelés vésiulesD qui sont responsles du on fontionnement d9une ellule vivnteF ne ompréhension pprofondie des modèles dynmiques et géométriques des vésiules et de l ellule en générlD peut onduire à une ompréhension iologique plus proE qulité des résultts et l durée de tritement doit être e'etuéF ge proE • Le ou est le résultt du mouvement rpide de l9ojet pr rpport lème devient enore plus évident dns les sènes omplexesD présentnt u pteur u file réglge de l9élirge qui onduisent à des temps d9exposition longs de l9ppreilY un grnd nomre d9ojetsF
	he ulity model uses n evidene term to determine the presene of trget he proposed models presented in ghpter R nd ghpter S enle the detetion oustiquesD rdrD ultrsoniques ou optiquesF ghque pteur possède des vnE fréquene temporelle inférieureF oilà pourquoi nous onsidérons une lrge gmme v9utilistion des vidéos pour l télédétetion pr stellite (n de déteter et de suivre • Les ombres peuvent être usées soit pr les ojetifs euxEmêmes ou
	oupled with ontrst distne mesure whih ims to orretly (t the ellipse over nd trking of moving ojets nd provide high level informtion out the ojets tges et des inonvénients et le hoix du type de pteur emrquer à ord d9un de fréquenes temporelles pour les vidéos onsidérées dns ette thèseF des ojets est une tâhe di0ileF hes dé(s sont reontrés tout u long de l hîne pr les ojets environnntsF in milieu urinD les omres peuvent être

the

  ext et θ int et θ = [θ ext , θ int ]FLe terme d'énergie interneF ve terme d9énergie interne est onstitué d9un ensemle de ontrintes destinées à une détetion orrete des ojets (n de filiter le suiviF ges ontrintes ilent l disposition des ojetsF r exempleD l ohérene géometrique et physique doit être mintenue tout u long de l séqueneF xous dé(nissons trois termes énergétiques internes omme suitX• Le modèle dynamiqueF ne propriété rtérissnt le suivi @pr opposition à des détetions individuelles pr imgeA est que dns l pluprt des sD les trjetoires des iles sont lissesF gel permet de fvoriser des on(gurtions où les iles présentent un mouvement dérit pr un modèle dynmiqueF our nos expérienesD nous supposons que les ojets suivent un modèle dynmique à vitesse onstnteF fien que e modèle de mouvement soit reltivement simpleD il est ssez générl pour ouvrir une grnde vriété d9pplitions de suiviF ge terme fvorise l détetion de iles lorsque le terme d9tthe u données à une file vleurD mis que le modèle dynmique indui l9existene de l ileY

	B.5. Publications	163
	données ont été onstruits à l minD et ensuite vlidés pr un expertF v prtie 5.2. développe une nouvelle pprohe de progrmmtion linéire l9intertion entre les ojets dns un dre unique et à trvers l totlité de l
	hes ensemles de données seondires provennt du domine iologique ont égleE pour estimer les prmètres du modèle dns le dre de proessus de séquene d9imges onsidéréeF v9énergie totle peut être érite omme l somme de
	ment été nlysés dns ette thèseF ges ensemles de données ont été fournis pr pointsY es deux termesX
	l9snstitut steur à ris et l9snstitut gurie à risF in outreD un logiiel grtuit U θ (X, Y) = U ext θext (X, Y) + U int θ int (X). @gFQA v prtie 5.3. propose de nouveux noyux de perturtion spéi(queE nommé sy et développé pr l9nité d9enlyse untittive de l9snstitut steur à ment justés pour le prolème de suivi des ojets multiples à utiliser dns ves veteurs de prmètres des termes d9énergie externe et interne sont respetiveE ris été utilisé pour réer utomtiquement des imges iologiques synthétiques et des vidéos ve une vérité terrin ssoiée à hque séquene pour évluer l qulité des modèles proposés dns ette thèseF B.4 Organisation de thèse et contributions gette thèse est orgnisée omme suitX • Le chapitre 1 donne un perçu de l thèseD dérit les données utiliséesD souligne l motivtion de e trvil et résume les ontriutionsY • Le chapitre 2 derit l9étt de l9rt des pprohes liées ux prolèmes de détetion et de suivi d9ojetsY • Le chapitre 3 présente les proessus pontuels omme un dre pproprié pour l détetion d9ojetsD insi que pour le suiviF v9étt de l9rt pour des tehniques d9estimtion de prmètres et de simultion spéi(quement dpE téess à e dre est ussi déritY le shém d9optimistion twgwgF • Le chapitre 6 onlut l thèse et dérit les perspetives potentielles de e trvilF B.5 Publications ves ppiers suivnts ont été puliés ou soumis à prtir des trvux ontenus dns e mnusrit de thèseX • Dans des revuesX F gr iunD wF yrtnerD et tF eruiF e sptioEtemporl mrked point proess model for joint detetion nd trking of moving ojetsF ousmis à IEEE Transactions on Image ProcessingD PHISY F gr iun et tF eruiF nsupervised mrked point proess model for ot extrtion nd ounting in hrors from high resolution optil remotely sensed imgesF Revue Francaise de Photogrammétrie et Télédé-ment θ • Persistance de l'étiquetteF e(n de fire l distintion entre des trjetoires tectionD volF PHUD ppF QQERRD PHIRY distintesD une étiquette est joutée à l mrque de hque ellipseF gette étiE • Le chapitre 4 introduit les premières ontriutions importntes de ette • Dans des conférences nationales et internationalesX quette peut être onsidérée omme un identi(teur de trjetoireF hi'érentes thèseF sl fournit le modèle développé pour l détetion et le omptge des teux dns les zones portuiresF ves ontriutions de e hpitre peuvent se résumer omme suitX étiquettes signi(ent di'érentes trjetoiresF einsiD le nomre d9étiquettes doit F gr iunD wF yrtnerD nd tF eruiF ousmis à siii gomputer ision être étroitement lié u nomre de trjetoires dns l9ensemle de donnéesF nd ttern eognitionD eD PHITY sdélementD le grnd nomre de iles dispersées à trvers l séquene d9imges
	F gr iun nd tF eruiF ousmis à siii snterntionl ymposium on doit être ssingé à un ssez petit nomre d9étiquettesF r onséquentD nous v prtie 4.1. présente un modèle mélioré de proessus de point mrqué fiomedil smgingD gzeh epuliD PHITY utilisons e terme pour limiter le nomre d9étiquettesD fvorisnt insi les onE mis u point pour l détetion des teux et le ompre u modèle F gr iunD wF yrtnerD et tF eruiF toint detetion nd trking of movE (gurtions ve un petit nomre d9étiquettes distintesY préédemment présenté pr fen rdj et lF PHIHY v prtie 4.2. dérit les tehniques d9estimtion des prmètres utilisées ing ojets using sptioEtemporl mrked point proessesF IEEE Winter • L'exclusion mutuelleF v mnipultion de l ollision entre ellipses ou le Conference on Applications of Computer VisionD eD PHISY hevuhement dns une trme donnée est un spet ruil lors de l détetion pour de(nir les prmètres du modèleY F gr iun et tF eruiF owrds e0ient simultion of mrked point et le suivi des ojetsF hns nos modèlesD une pénlité in(nie est ttriuée à v prtie 4.3. propose une nouvelle mise en ÷vre prllèle du shém d9optimistion twgwg lrgement onnuF proess models for ot extrtion from high resolution optil remotely une on(gurtion qui ontient des ellipses qui se hevuhent plus qu9un seuil sensed imgesF International Geoscience and Remote Sensing Sympo-(xé à l9vneF einsiD l proilité de séletion d9une on(gurtion de e type siumD gndD PHIRY est égle à zéroF ge terme d9énergie est utilisé pour limiter le nomre d9ellipses • Le chapitre 5 introduit l ontriution priniple de ette thèseF hns e F gr iun et tF eruiF nsupervised mrked point proess model for qui ouvrent le même emplement dns une trme donnéeF hpitreD nous dérivons un modèle de proessus de point mrqué spéi(E quement développé pour le suivi d9ojets multiplesF ves ontriutions de e ot extrtion in hrors from high resolution optil remotely sensed ne desription détillée de hque terme peut être trouvée dns l prtie SFIFIF imgeF International Conference in Image ProcessingD eustrlieD PHIQY hpitre peuvent se résumer omme suitX F gr iunD wF yrtnerD et tF eruiF sntegrting twgwg nd ulmn Terme externe d'énergieF xous vons développé deux pprohes lterntives
	v prtie 5.1. introduit un nouveu modèle et présente une desription pilters for multiple ojet trkingF GRETSI -Traitement du Signal et pour relier une on(gurtion ux donnéesX le Quality model et le Statistical
	modelF	pprofondie des ontrintes et des restritions de e modèleY des ImagesD prneD PHISY

  i est un prmètre de tempérture qui tend vers zéro lorsque i tend vers l9in(niF i T emp i diminue logrithmiquementD lors X i tend vers un optimiseur glol f θ,i F hns l prtiqueD ependntD une loi logrithmique n9est ps eptE leD pr onséquentD une loi géométrique est utilisée à l ple pour diminuer l tempertureF v9éhntillonneur twgwg été intégré dns un shém de reuit simulé pour de meilleures performnesF xénmoinsD el ussi onduit à un oût de lul rueF our fire fe à e prolèmeD nous vons proposé deux nouvelles pprohesX• L'intégration des perturbations de Kalman dans le RJMCMCF ves (ltres ont prouvé qu9ils pouvient fournir des performnes de suivi reltiveE ment rpides et peu outnteD en prtiulier pour le suivi d9une seule ileF ves propriétés des (ltres séquentiels peuvent être e0ement exploités sur le pln de l9éhntillonnge twgwgF ve (ltre de ulmn est utilisé pour générer des propositions de perturtions plus pertinentes qui sont ensuite évluées en utilisnt un tux d9epttion de qreen ppropriéF xous introduisons un noyu de nissne et de mort dédiéD à l9ide des mouvements générés pr le (ltre de ulmnF hes propositions de perturtions meilleures ugmentent l proilité de l9éhntillonneur twgwg de hoisir une telle perturtion quiD à son tourD onduit à une onvergene plus rpideF lus de détils peuvent être trouvés dns l prtie SFQFRF• La mise en ÷uvre paralléle du RJMCMCF ne mise en ÷uvre prlléle reherhe montre que les modèles de proessus pontuels mrqués peuE vent être utilisés à pour le suivi d9ojetsF ne nlyse qulittive des résultts révèle des performnes élevées de détetion et de suivi pour toutes les vidéos présentées dns ette thèseF ves vntges de ette pprohe sont disutés dns l setion suivnteF D.2 Advantages de méthode proposée ves modèles présentés dns ghpitre R et ghpitre S permettent l détetion et le suivi d9ojets en mouvement et fournissent des informtions de hut niveu sur les ojets tels que leur trjetoireD leur tilleD leur forme et leur orienttionF ve terme d9énergie ouplé ve le proessus d9optimistion en th permet l déteE tion d9ojets filement ontrstés ou des ojets temporirement olues tout en mintennt des trjetoires ohérentesF gette pprohe est utomtique et permet de struturer l9extrtion des trjetoires sur l se des intertions entre les ojetsF ves intertions omplexes entre les ojets peuvent être e0ement intégrés dns les modèlesF n vntge importnt de ette méthode est s roustesse u type de ruit et qulE ité des données utiliséesF in prtiulierD l9pprentissge des prmètres est e'etuée une seule fois pour hque ensemle de donnéesF ve modèle peut lors être ppliquée ve suès à hque séquene dns et ensemle de donnéesF xous expérimentons ette thèseD nous vons modélisé le prolème de suivi u niveu des ojetsF ves intertions entre es ojets sont dérites à l fois dns le domine sptil et temporelF v9informtion sémntique à propos de trjetoires d9ojets est ensuite réupérée pr le regroupement des ojets à prtir de leurs étiquettesF ne pprohe intéressnte serit de onevoir un modèle hiérrhique qui intègre à l fois les onE trintes de s niveu entre les ojets individuels et les ontrintes de hut niveu entre les trjetoiresF v mrque géométrique d9un point dns le proessus pontuel mrqué dépend de l9pplition onsidéréeF ves formes simples telles que des retngles ou des ellipses sont plus souhitles pour rtériser l géométrie des ojets r ils sont dérits

	D.2. Advantages de méthode proposée D.5. Conclusions	173 175
	où T emp est proposéeF eujourd9huiD les ordinteurs lssiques ont générlement plus de deux unités de tritement etD pr onséquentD les implémenttions prlE léles deviennent uiquitiresF ne ttention prtiulière doit être ordée (n de veiller à e que les perturtions prlléles soient onditionnellement indépendnte dns les deux dimensions sptiles insi que dns le tempsF ne desription détillée de l9éhntillonneur twgwg prlléle se trouve dns l prtie SFQFSF ves deux pprohes d9optimistion sont fondmentlement di'érentesD mis elles ont le même utX une simultion plus rpide de l fontion d9énergie totle dérite préédemmentF n spet importnt de suivi d9ojets multiples est non seulement l détetion onséutive des iles dns l séquene d9imgesD mis ussi l9étiquetge orret de l9pprohe proposée sont testées sur es séquenesF out les résultts sont dérits l9espe de l9imge en ellules indépendntes et d9e'etuer des perturtions méliortions onsidérles doivent être rélisées à et égrd pour rendre les modèles de l vidéoD plutôt que des trjetoires individuelles peut être intéressnt dns de on(gurtionX données iologiques peuvent être générés ve l vérité terrin ssoiéeD les limites sls préonisent un système de prtitionnement hiérrhique (n de diviser fréquene ve de petites imges et un nomre limité d9ojetsF r onséquentD des imges de mirosopieF n modèle pour estimer l densité des ojets tout u long xous vons nlysé deux pprohes di'érentes pour réétiqueter les ellipses dns une nomre d9exemples représenttifsF lus préisémentD puisque les ns d9essi des grndes méliortions en termes de temps de lul sur l version séquentielleF thèseD l performne en temps réel peut être otenue que pour les vidéos de sse les données stellitires ou l irultion près de l memrne ellulire dns les ontexteD l9étiquette de l9ellipse détermine l trjetoire à lquelle l9ojet pprtientF ves résultts pour hune des tégories de données sont illustrés à prtir d9un petit proposé une mise en ÷uvre prllèle de l9éhntilonneur sur q et otenu de ps à suivre e rythmeF hns les prmètres tuels tels que présentés dns ette ps ux véhiulesD elle peut églement inlure les piétons et le tr( mritime pour des iles (n que des trjetoires ohérentes puissent être extritesF hns notre PF Étiquetage déterministe pendant les mouvements de naissanceF gette pprohe de l9étiquetge onsiste à ttriuer l9étiquette d9une nouvelle ellipse sur l se de son voisingeF und une nouvelle ellipse est rééD une reherhe est e'etuée (n de trouver l9ojet le plus prohe dns son voisingeD et l vleur de l9étiquette de l nouvelle ellipse est mise égle à elle de l9ellipse representnt l9ojet le plus proheF v vitesse de lul otenue ouplée à l onne performne résultnte motivent l9utilistion d9une telle pproheF ne nlyse en profondeur de tous les noyux de perturtion utilisés est présentée dns les prties SFQFIF et SFQFPF RésultatsF ves modèles proposés ont été ppliqués à une grnde vriété d9ensemles de donnéesD qui peuvent être lssés omme suitX • Les bancs d'essai synthétiques utilisés par la communauté de traite-ment d'images biologiquesF lusieurs séquenes ont été générées ve le logiiel lire syD issu de l9nité d9enlyse untittive d9smges fiologiques de l9snstitut steur @w gx 3691AD dirigée pr tFEgF ylivoEwrinF ves séquenes d9imges ontiennent di'erents niveux de ruit et un nomre d9ojets vrileF di'érents niveux de ruit et un nomre di'érent d9ojetsF ves ojets peuvent entrer ou sortir de l région d9intérêt à tout moment et dns n9importe quel lieuF ges ensemles de données ont été lrgement utilisés pour nlyser les performnes de l méthodeD ompte tenu de l9existene des informtions de vérite terrin préisY • Des données biologiques réellesF ne séquene de données iologiques réellesD fournie grieusement pr tF lmeroD sg sfieD w IRR gxD snstitut gurie @fsset et lFD PHIRAD est utilisée pour montrer l9ppliilité de l méthode proposée pour des données réelles dns le ontexte iologiqueY • Des données satellitaires simuléesF ges ensemles de données sont des séquenes d9imges stellitires réelles qui ont été modi(ées pr eirus h8 pour ressemler à des imges produites pr un stellite géosttionnireF ne grnde vriété d9éhntillons est exploréeD des mers sttiques ux mers l9éhntilonneur twgwg été fite pr erdié nd vfrge PHIPF sts ont xénmoinsD dns une ère de tritement en temps réelD es méthodes ne prviennent désirles que les trjetoires elles mêmesF v notion de irultion ii ne se limite détetés et suivis u ours de l séquene d9imgesF RJMCMCF ne tenttive réussie pour roître l9e0ité de lul de d9élérer le proessus d9optimistion onsidérlement réduit le temps de lulF l irultionD des informtions sur l densité de l irultion peuvent être plus temporellesD d9un seul type d9ojet à plusieurs types d9ojets qui doivent être SF La mise en ÷uvre parallèle et ecace de l'échantillonneur que nous vons proposée dns e trvil (n de réduire l9espe de reherhe et en mouvementF gependntD dns ertines pplitions telles que l surveillne de en mouvementD des hutes fréquenes temporelles ux des sses fréquenes ojetsF hns des portsD les teux sont nrés très prohes les uns des les utresF r onséquentD deux ou plusieurs teux voisins sont générlement détetés omme un ojet de grnde tilleF fen rdj et lF PHIH ont proposé un modèle pour e prolème d9extrtion lorsque tous les ojets ont l même orienttionF xous onstruisons sur le modèle présenté pr fen rdj et lF PHIH et développons un modèle plus générlD et e0eD pour ompter les teux dns des ports indépendmment de leur orienttionF in prtiulierD nous értons les ontrintes qui rendent le modèle de fen rdj et lF PHIH trop restritif et introduisons des heuristiques simples pour méliorer l qulité du résulttY QF Estimation automatique des paramètresF v performne des modèles de proessus de point mrqué pour une pplition spéi(que dépend d9un hoix ien dpté et e0e des prmètresF ves modèles présentés dns ette thèse sont une ominison linéire de plusieurs termes d9énergie pondérésF ges poids sont générlement di0iles à de(nir mnuellement r ils mnquent une interpréttion physique direteF xous vons développé une tehnique uE tomtique d9estimtion des prmètres fondée sur l progrmmtion linéire qui peut être e0ement utilisée pour dé(nir les prmètres de es modèlesY RF Un échantilonneur RJMCMC intégré avec des mouvements de KalmanF ves dynmiques de twgwg sont utilisées pour simuler les modE èles présentés dns ette thèseF ve twgwg est un éhntillonneur itértif qui utilise un ensemle de noyux de perturtion pour simuler une hîne de wrkov qui onverge vers l distriution souhitéeF get éhntillonneur de très onnes propriétés de onvergene et 9est pour el qu9il est d9un grnd neur hyride en intégrnt l9éhntilloneur twgwg ve des mouvement de vient ve une hrge de lul intrinsèqueF v9utilistion d9heuristiques e0es xous vons onçu nos modèles pour extrire des trjetoires individuelles des ojets souhitéeY mthémtique des proessus pontuels mrqués dns lequel e trvil est développé s trjetoireF initer le hîne de wrkov à onverger plus rpidement vers l distriution out d9ordD l9e0ité de lul demeure un spet prolémtiqueF ve dre préutions doivent être prises pour préserver l lsse d9un ojet tout u long de type de ulmnF ves mouvements issus du (ltre de ulmn sont utilisés pour envisgée pour fire l distintion entre di'érentes lsses d9ojetsF gependntD des D.3 Inconvénients des méthodes proposées férentes formes dns une imgeF ne extension pour les données vidéo peut être prix d9un oût de lul élevéF xous vons proposé un nouveu éhntillonE et lF PHIH proposent un proessus multiEmrqué pour extrire des ojets de difE intérêt pour l ommunuté sienti(queF xénmoisD es propriétes sont u moinsD l9évolution vers l9utilistion de plusieurs formes serit souhitleF vfrge été identi(és u ours de e trvil de thèseF ojets d9intérêt @pr exempleD les voituresD les teuxD les vésiulesD etFAF xénE fien que les vntges d9un tel proédé soient solidesD plusieurs inonvénients ont uniquement des ellipses u long de e trvil pre qu9elles intègrent l pluprt des similires et montrons que l qulité du résultt n9est ps signi(tivement 'etéeF otenir des résultts de hute qulitéF gependntD l9utilistion d9un tel modèle se révèle être limittif en termes d9pplitionsF r onséquentD un modèle de ile plus éloré qui puisse intégrer plus d9informtion sur l9pprene des iles serit souhitleF D.4 Perspectives pr un petit nomre de prmétresF in prtiulierD nous vons hoisi d9utiliser D.5 Conclusions ge mnusrit présente seulement une prtie des trvux entrepris u ours des es trois dernières nnéesF xous vons eu l9osion d9explorer des pprohes et théories di'érentes que elles dérites iiD d9expérimenter ve une grnde quntité de données et de proposer systémtiquement de nouvelles solutions à prtir des impsses et tenttives de modélistion éhouéesF in tnt que tellesD es dernières ge trvil de églement ve l9ide des mêmes prmètres sur plusieurs ensemles de données nnées ont été très instrutivesF
	IF Perturbations de ssion et fusion des trajectoiresF n mouvement dns l prtie SFRF indépendntes dns es ellulesF r onséquentD des di0ultés surgissent pour de proessus de point mrqué ppropriés pour le suivi d9ojets multiples à l9venirF telles pplitionsF
	de ssion onsidère une trjetoire donnée et propose de l sinder en deux extrire orretement les ojets u ord de l elluleF xous vons proposé une n utre inonvénient des modèles proposés est dû u modèle d9ellipse rillnte in(nD nous vons proposé plusieurs fçons d9méliorer l performne de
	trjetoires di'érentes à un point de prtge (xéF n mouvement de fusion nouvelle mise en ÷uvre multiE÷rs du twgwg qui mintient une mémoire utilisé pour rt ¡ 9eriser une ileF ge modèle de ile est su0sment simple pour l9éhntilonneur twgwgF wême insiD le proessus d9optimistion devrit être
	Y) T emp i propose d9unir deux trjetoires distintes et de réétiqueter les ellipses de sorte prtgée entre les proesseurs de sorte que les perturtions dns une ellule être e0ement inorporé dns le modèle de proessus de pointF he plusD pour enore mélioré pour rendre es modèles ompétitifs ve l9étt de l9rt existnt sur @gFTA qu9elles forment une seule trjetoire plus longueF sont rélisées en tennt ompte de son viosingeF les pplitions présentées dns ette thèse un modèle simple de ile su0t pour des lgorithmes de suivi en temps réelF

hns

  rkinsonD F qerstenD nd vF tokF eelerted deline in the erti se ie overF Geophysical research lettersD QS@IAD PHHVF @gited on pges II nd ISQFA uF gsilléryD wFqFfF flumD yFiF qggiottiD nd yF prnçoisF epproximte fyesin gomputtion efg in prtieF Trends Ecol. Evol.D PSXRIH!RIVD PHIHF @gited on pge RPFA xF hll nd fF riggsF ristogrms of oriented grdients for humn detetionF Proc. CVPRD IXVVT!VWQD PHHSF @gited on pges PP nd IPUFA qFfF hntzigF viner progrmmingF Proc. of Symposium on Modern Calculating Machinery and Numerical MethodsD ISXIV!PID IWRVF @gited on pge IHQFA qFfF hntzigF Linear programming and extensionsF rineton niversity ressD IWTQF @gited on pge IHQFA

	Bibliography
	qF fertorelleD eF fenzzoD nd F wonF efg s )exile frmework to estimte tFgF gomisoD gFvF
	demogrphy over spe nd timeX ome prosD some onsF Mol. Ecol.D IWXPTHW!
	PTPSD PHIHF @gited on pge RPFA
	FwF fhngle nd FF hurhF righ performne sp feture lssi(tion of
	r stellite imgery for disster mngementF Proc. IGARSSD pges QSUR!QSUUD
	PHIRF @gited on pges II nd ISRFA
	sF fiedermnF eognitionEyEomponentsX theory of humn imge understndingF eddisonEesleyD IWWWF @gited on pge UVFA Psychol. Rev.D WRXIIS!IRUD IWVUF @gited on pge PQFA
	wF entoniniF Compression des images et des viéos numériques. Dix années des eF foisunon nd tF eruiF istimtion of the weight prmeter with eiw recherches au CNRSF rilittion à diriger des reherhes @rhAD niversité de for mrked point proesses pplied to ojet detetionF Proc. EUSIPCOD pges xie E ophi entipolisD PHHQF @gited on pges IR nd ISTFA PIVS!PIVWD PHIRF @gited on pge RRFA
	F frEhlom nd F portmnnF Tracking and Data AssociationF edemi ressD F fouwmnsF eent dvned sttistil kground modeling for foreground n hiegoD IWVVF @gited on pge IIUFA detetionX systemti surveyF Recent patents on computer scienceD R@QAXIRU!
	eF fssetD tF foulngerD F fouthemyD gF uervrnnD nd tF lmeroF vEvoqX e IUTD PHIIF @gited on pge PSFA
	esile egmenttion wethod with eutomti le eletion nd vol hreshE vF freimnF ndom forestsF Machine learningD RS@IAXS!QPD PHHIF @gited on olding epplied to sp wirosopyF ISBI -2014 IEEE International Symposium on Biomedical ImagingD pges SQQ!SQTD PHIRF @gited on pges PD TD IPID IPSD IPT pge PTFA
	nd IUHFA eF fugeu nd F erezF rk nd utX simultneous trking nd segmenttion of
	multiple ojets with grph utsF EURASIPD PHHV@QAXTHQ!TIWD PHHVF @gited on wFeF feumontF epproximte fyesin gomputtion in evolution nd eologyF Annu. Rev. Ecol. Evol. Syst.D RIXQUW!RHTD PHIRF @gited on pge RPFA pge PTFA
	F gettD F gollingsD uF ringeeD hF wprlneD nd F iolingF pineEsle
	wFeF feumontD eFF hngD nd hFt fldingF epproximte fyesin gomputE monitoring of omplex environments using remotely sensed erilD stellite nd tion in popultion genetisF GeneticsD ITP@RAXPHPS!PHQSD PHHPF @gited on pge RPFA other sptil dtF Proc. of ISIDFD pges I!SD PHIIF @gited on pges IP nd ISRFA
	iF felikovD F heliginnisD F otooD wF eljriD nd rFEF voidlF e survey of highE tF gnnyF e omputtionl pproh to edge detetionF IEEE Transactions of
	level prllel progrmming modelsF ehnil reportD reriotEtt niversityD PAMID V@TAXTUW!TWVD IWVTF @gited on pge TVFA
	PHIQF @gited on pge UVFA F goD tF vnD F nD nd F viF uv feture sed vehile detetion nd trking
	F fen rdjD pF ghtelinD F hesomesD nd tF eruiF rmeter estimtion for in irorne videosF Proc. ICIGD PHIIF @gited on pges PI nd PSFA
	mrked point proess within frmework of multidimensionl shpe extrtion qF geleux nd tF hieoltF he iw lgorithmX proilisti teher lgorithm from remote sensing imgesF Proc. of ISPRS Conference on Photogrammetry derived from the iw lgorithm for the mixture prolemF Computational statistics Computer Vision and Image AnalysisD sssD PHIHF @gited on pges IVD IWD PRD TQD TRD IRUD IRVD ITID ITP nd IUPFA quarterlyD SS@RAXPVU!QIRD IWVSF @gited on pges RQ nd UQFA
	F ghllD wFF worelndeD hF wusikiD nd FtF ivnsF Fundamentals of object F fen rdjD pF ghtelinD F hesomesD nd tF eruiF istimtion des prmètres de modèles de proessus pontuels mrqués pour l9extrtion d9ojets en imgerie trackingF gmridge niversity ressD PHIIF @gited on pge PVFA
	sptile et érienne hute résolutionF eserh eport EUQSHD sxseD PHIHF fF ghpmnD qF tostD nd F vn der sF Using OpenMP: portable shared memory
	@gited on pges IV nd ITHFA parallel programmingF ws ressD PHHUF @gited on pge UVFA
	tF ferlzD pF pleuretD nd F puF wultiple ojet trking using )ow liner proE FeF gherghi nd FF heikhF woving ojet detetion using imge registrtion
	grmmingF IEEE Workshop on Performance Evaluation of Tracking and Surveil-for moving mer pltformF Proc. ICCSCED PHIPF @gited on pges PID PS
	lanceD pges I!VD PHHWF @gited on pge IHRFA nd PTFA

  khronos.org/registry/cl/ specs/opencl-1.1.pdfD PHIQF eessedX xovF PHIQF @gited on pge UWFA qF uitgwF xonEqussin stte spe modeling of nonEsttionrityF Journal of American Statistical AssociationD VP@RHHAXIHQP!IHRID IWVUF @gited on pge PWFA FgF uoopmnsF Actively analysis of production and allocationF tohn iley nd onsD IWSIF @gited on pge IHQFA eFrF uydd nd fFpF lterF he strtegies of terrorismF International securityD QI @IAXRW!UWD PHHTF @gited on pges II nd ISQFA gF vosteD F hesomesD nd tF eruiF oint proesses for unsupervised line network extrtion in remote sensingF IEEE Trans. PAMID PU@IHAXISTV!ISUWD PHHSF @gited on pges IQD PR nd ISSFA pF vfrgeD qF qimel9prD nd F hesomesF qeometri feture extrtion y multiEmrked point proessF IEEE Trans. PAMID QP@WAXISWU!ITHWD PHIHF @gited on pges SWD ISH nd IURFA pF vfrgeD F uerivenD wF fredifD nd rF uF ryrid multiEview reonstrution y jumpEdi'usionF Proc. CVPRD pges QSH!QSUD PHIHF @gited on pge SWFA F vnduF Surveillance or Security? The risks posed by new wiretapping technolo-giesF he ws ressD PHIIF @gited on pges II nd ISQFA hF vvigneD F hliD F yuyngD nd F hengF nsupervised lssi(tion nd lustering of imge fetures for vehile detetion in lrge sle eril imgesF Proc. FUSIOND PHIHF @gited on pge PIFA iFvF vehmnn nd tF F omnoD editorsF Testing statistical hypotheses, 3rd editionF pringerD PHHVF @gited on pges WU nd WVFA fF veieD uF hindlerD xF gornelisD nd vF vn qoolF goupled ojet detetion nd trking from stti mers nd moving vehilesF IEEE Trans. PAMID QH@IHAX ITVQ!ITWVD PHHVF @gited on pges PR nd IHVFA F vi nd F friggsF eutomti extrtion of rods from high resolution eril nd stellite imges with hevy noiseF Proc. of the 6th International Conference on Geographic Information SystemsD PHHWF @gited on pges TV nd TWFA F viD gF rungD nd F xevtiF verning to ssoiteX ryridfoosted multiEtrget trker for rowded seneF Proc. CVPRD PHHWF @gited on pges PV nd PWFA F viuD gF ghenD F henD nd F ouF hetetion of smll ojets in imge dt sed on the nonliner prinipl omponent nlysis neurl networkF SPIE Optical EngineeringD RRD PHHSF @gited on pge PTFA xFuF vogothetis nd hFvF heinergF isul ojet reognitionF Annu. Rev. Neurosci.D IWXSUU!TPID IWWTF @gited on pge PQFA 184 Bibliography FsF vukowski nd pFtF ghronneuF yntheti perture rdr nd seh nd resueF Proc. IGARSSD SXPQUR!PQUTD PHHHF @gited on pges II nd ISRFA F vuoD pF viuD F viuD nd F ngF ttionry vehile detetion in eril surveilE lne with eF Proc. ICIDTD PHIPF @gited on pges PI nd PTFA F whlerF wultitrget fyes (ltering vi (rstEorder multi trget momentsF IEEE Trans. on Aerospace and Electronic SystemsD QW@RAXIISP!IIUVD PHHQF @gited on pge QHFA F whlerF Statistical Multisource-Multitarget Information FusionF xorwoodD weD eX erteh rouseD PHHUF @gited on pge QIFA F whlerF rh (lters of higher order in trget numerF IEEE Trans. on Aerospace and Electronic SystemsD RQ@RAXISPQ!ISRQD PHHUF @gited on pge QIFA F whlerF Advances in Statistical Multisource-Multitarget Information FusionF xorE woodD weD eX erteh rouseD PHIRF @gited on pge QIFA tFEwF wrinF Bayesian core: a practical approach to computational Bayesian statis-ticsF pringerD PHHUF @gited on pges IU nd ITHFA tF wteu nd pF wontesF seudoElikelihood inferene for qis proesses with exE ponentil fmilies through generlized liner modelsF Statistical Inference for Stochastic ProcessesD R@PAXIPS!ISRD PHHIF @gited on pge RPFA FqF wttsonD fFeF ndersD nd fF wssingillF Patterns for parallel programmingF eddisonEesleyD PHHSF @gited on pges UU nd UVFA xF wvughlinD tF wrtinez hel ionD nd F willerF inhning liner progrmming with motion modeling for multiEtrget trkingF Proc. of WACVD pges UI!UUD PHISF @gited on pge IHRFA F wertonF yption priing when underlying stok returns re disontinuousF Journal of Financial EconomicsD QXIPS!IRRD IWUTF @gited on pge SVFA xF wetropolisD wF osenluthD eF ellerD nd iF ellerF iqutions of stte luE ltions y fst omputing mhinesF Journal of Chemical PhysicsD PIXIHVU!IHWID IWSQF @gited on pges RS nd THFA eF wilnD F othD nd uF hindlerF gontinuous energy minimiztion for multitrget trkingF IEEE Trans. PAMID QT@IAXSV!UPD PHIRF @gited on pges ID IPP nd IPQFA F wüller nd wF wüllerF gesX smproving mou)ge ssessment with ssisE tne methodsF Proc. of SPIED UTTPD PHIHF @gited on pges IR nd ISUFA eFxF xetrvli nd fFqF rskellF Digital pictures: Representation, Compression and StandardsF pringerD IWWSF @gited on pges IR nd ISTFA FF xguyenD rF qrnerD rF fishofD nd fF qruerF ynEline oosting for r detetion from eril imgesF Proc. RIVFD PHHUF @gited on pge PPFA xidiF ghe oneF http://www.nvidia.com/object/cuda_home_new.htmlD PHIQF eessedX xovF PHIQF @gited on pge UVFA F yhD F ussellD nd F stryF wrkov ghin wonte grlo ht essoition for generl multipleEtrget trking prolemsF Proc. CDCD IXUQS!URPD PHHRF @gited on pge PWFA ypenwF ypenw QFH pei(tionF http://www.openmp.org/mp-documents/ spec30.pdfD PHIQF eessedX tune PHIQF @gited on pge UVFA wF yrtnerF Processus ponctuels marqués pour l'extraction automatique de caricatures de batiments à partir de modèles numériques d'élévationF hh thesisD niversité de xie E ophi entipolisD PHHRF @gited on pges IQD IVD PRD RVD SID SPD USD ISS nd ITIFA wF yrtnerD F hesomesD nd tF eruiF en dptive simulted nneling ooling shedule for ojet detetion in imgesF eserh eport TQQTD sxseD PHHUF @gited on pge TIFA

de fusses lrmes puisque les ojets sttiques semlent églement se dépler en rison du déplement de l mérF pigure fFI @dA montre un exemple d9une zone urine denseF v sène ontient près de 100 véhiulesF ghque véhiule dns l petite fenêtre été étiqueté mnuelleE ment ve une rré rougeF v9étiquetge mnuel est utilisé omme une informtion de vérité terrin pour évluer l performne de l9lgorithme utomtique proposé de détetion et de suivi proposéF n lgorithme e0e devrit diminuer de fçon signi(tive le temps de tritement requis pr rpport à un opérteur huminD tout en mintennt un hut niveu de préisionF lusieurs pprohes ont été dévelopées dns le pssé pour relever les dé(s présentés iEdessus et sont présentés dns l ghpitre PF xénmoinsD il y un grnd potenE tiel pour méliorer l performne pr rpport ux méthodes tuelles à l fois en termes de roustesse et de temps de tritementF

Acknowledgments

A.4 International Summer School

Summer School on Topics in Space-Time Modeling and InferenceD PUEQI wy PHIQ t elorg niversityD henmrkF @A @A @A @dA pigure fFIX n exemple pour hque dé( de l détetion d9ojet et le suivi à prE tir d9un pteur stellitire donné @ eirus h8AF @A v petite tille de l9ojet onduit à une modélistion de l9pprene limitéeF v reonnissne d9un ojet individuel est di0ileF ves ords entre les deux ojets sont )ousF @A ves omres projetées pr de grnds immeules ugmentent l di0ulté de détetion d9ojets en rison du file ontrste entre les ojets et l9rrièreEplnF ves omres dues ux utous et mions se déplent ve l9ojet et peuvent être identi(ées pr erreur ommes des ilesF @A ve mouvement de l mér produit des grndes distorsions dns des trmes onséutivesF @dA ixemple de zone urine denseF Résumé étoé v9ojetif prinipl de ette thèse est d9nlyser l9utilistion des ontrintes géometriques dns les méthodes de détetion et de suivi d9ojets vi des proessus pontuels mrquésF gette pprohe permet l9inlusion de deux types de ontrintes géometriquesX l forme de l ile @omme nous trintons priniplement d9ojets friqués pr l9hommeD leur forme peut générlement être e0ement estimée à l9ide d9une forme prmétrique simpleA et l dispersion géometrique des ilesF xous dé(nissons un proessus pontuel mrqué d9ellipses qui se rtérise pr l loE listion sptile et temporelle des ellipsesD leur demiEgrnd xe et demiEpetit xeD leur ngle pr rpport à l9horizontle et leur étiquetteF xous utilisons l9étiquette omme un ttriut permettnt de di'érenier les trjetoires distintes @pr exemE pleD toutes les ellipses ve l même étiquette forment une seule trjetoireAF xous utilisons une fmille de proessus de qis pour dé(nir l fontion de densité de proilité de nos modèles omme suitX

on(gurtion de toutes les ellipses à temps tY

• Y représente le ue d9imges 3hY

• θ est le veteur de prmètresY Y) µ(dX) est l onstnte de normlistionD Ω étnt l9espe des on(gurtions et µ(•) étnt l mesure de l9intensité du proessus de oisson de référeneY

• U θ (X, Y) est le terme d9énergieD qui dépend des prmetresF in utilisnt le ritère du mximum posteriori @weAD l on(gurtion l plus prole des ojets orrespond u minimum glol de l9énergieX

@gFPA v fontion d9énergie est divisée en deux prtiesX un terme d9énergie externeD U ext θext (X, Y) qui détermine à quel point l on(gurtion orrespond à l séquene d9entréeD et un terme d9énergie interneD U int θ int (X)D qui intègre les onnissnes sur