Dr Gabriele Moser

Prof Josiane Zerubia

Dr Roberto Sacile

Keywords:

Hierarchical joint classication models for multi-resolution, multi-temporal and multi

Contents

Introduction 5

1 Remote sensing and image analysis 9 IFI emote sensing system F F F F F F F F F F F F F F F F F F F F F F F F F F IH IFP inergy soures nd rdition priniples F F F F F F F F F F F F F F F F IP IFQ sntertions with the tmosphere F F F F F F F F F F F F F F F F F F F F IQ IFR inergy intertions with irth surfe fetures

F F F F F F F F F F F F IS IFS emporl domin F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IT IFT tellites nd sensors properties F F F F F F F F F F F F F F F F F F F F F
F F F F F F F F F F F F F F F F F F F F F F F F RH QFIFP he otts model F F F F F F F F F F F F F F F F F F F F F F F F F RI QFP qenerl hrteristis of wrkovin imge models F F F F F F F F F F RI QFQ snferene nd fyesin estimtion F F F F F F F F F F F F F F F F F F F RR QFQFI fyesin estimtors F F F F F F F F F F F F F F F F F F F F F F F RS QFQFP yptimiztion pprohes F F F F F F F F F F F F F F F F F F F F F RT QFR qrphil representtion of wrkovin models F F F F F F F F F F F F F RV QFRFI hependeny grph F F F F F F F F F F F F F F F F F F F F F F F F RW QFRFP qrphil interprettion F F F F F F F F F F F F F F F F F F F F F RW QFS guslity F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F
SFI wultiEtemporl struture F F F F F F F F F F F F F F F F F F F F F F F F F UR SFP wultitemporl ww inferene F F F F F F F F F F F F F F F F F F F F F F UT SFPFI ime t = 0X singleEtime wwF F F F F F F F F F F F F F F F F F UV SFPFP ime t = 1X (rst topEdown pssF F F F F F F F F F F F F F F F F UW SFPFQ ime t = 1X ottomEup pssF F F F F F F F F F F F F F F F F F F UW SFPFR ime t = 1X seond topEdown pssF F F F F F F F F F F F F F F VH SFPFS foth timesX omintion with wwhF F F F F F F F F F F F F F VH SFQ ixelwise lssEonditionl hps nd trnsition roilities F F F F VP SFQFI ixelwise lssEonditionl hps F F F F F F F F F F F F F F F F VP SFQFP rnsition proilities F F F F F F F F F F F F F F F F F F F F F VR SFR ixperimentl results F F F F F F F F F F F F F F F F F F F F F F F F F F F
F F F F F F F F F F F F F F F F F F F F F F F F F F IHP TFPFP ixperimentl results F F F F F F F F F F F F F F F F F F F F F F F IHS TFQ he seond proposed multiEsensor hierrhil model F F F F F F F F F IHW TFQFI wethodology F F F F F F F F F F F F F F F F F F F F F F F F F F F IHW TFQFP ixperimentl results F F F F F F F F F F F F F F F F F F F F F F F IIH
List of Tables IFI win hrteristis of the léides systemF F F F F F F F F F F F F F F PI IFP win hrteristis of the qeoiye systemF F F F F F F F F F F F F F F PP IFQ pundmentl hrteristis of gu opertionl modesF F F F F F F F PR PFI wultiEtemporl tehniques t the feture level using optil imgesF F QR PFP wultiEtemporl tehniques t the feture level using e imgesF F F QS PFQ wultiEtemporl tehniques t the deision level imgesF F F F F F F F F QT SFI xumer of trining nd test smples on the pnhromti pixel lttie of the léides imge @ITHH xIHHH pixelsA quired in PHIQ F F F F F F VU SFP glssi(tion uries on the test set of the léides dtsetX lss uriesD overll uryD nd omputtion timeF ixperiments were onduted using one @ITHHxIHHHA imge t level HD one @VHHxSHHA imge t level I nd four @RHHxPSHA nds t level P on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF F F F F F F F F WT SFQ glssi(tion uries on the test set of the qeoiye dtsetX lss uriesD overll uryD nd omputtion timeF ixperiments were onduted using one @ITHHxVHHA imge t level HD one @VHHxRHHA imge t level I nd one @RHHxPHHA nds t level P on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF F F F F F F F F WT TFI glssi(tion uries for the ortEuErine dtsetF ixperiments were onduted on n sntel iU qudEore @PFRH qrzA VEqfEew TRE it vinux systemF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IHV TFP glssi(tion uries of resultsF ixperiments were onduted on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF F IIS UFI esults otined using the léides dtsetX lss uriesD overll uryD nd omputtion timeF F F F F F F F F F F F F F F F F F F F F IPV

List of Figures

IFI he irth oservtion systemF F F F F F F F F F F F F F F F F F F F F F II IFP en eletromgneti wveF F F F F F F F F F F F F F F F F F F F F F F F F IP IFQ he eletromgneti spetumF F F F F F F F F F F F F F F F F F F F F F F IQ IFR petrl hrteristis nd tmospheri e'etsF F F F F F F F F F F F F IR IFS fsi intertions etween eletromgneti energy nd irth surfe fetureF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IS IFT petrl re)etne of vegettion nd wterF F F F F F F F F F F F F F F IT IFU petrl re)etne of leves F F F F F F F F F F F F F F F F F F F F F F F IT IFV he growth of the numer of oservtion systems operting in oth optil nd mirowve spetrum F F F F F F F F F F F F F F F F F F F F F IW IFW he Q quisition modes of gywyEkywed sensor F F F F F F F F F PR IFIH he quisition modes of drstEP sensor F F F F F F F F F F F F F F PS PFI qenerl ht fusion rhitetureF F F F F F F F F F F F F F F F F F F F F PW PFP wultiEtemporl method tegoriztionF F F F F F F F F F F F F F F F F F QI PFQ qenerl shem of dt preEproessingF F F F F F F F F F F F F F F F F F QQ QFI yne dimensionl ssing modelF F F F F F F F F F F F F F F F F F F F F F F RI QFP ystem of spins in the otts modelF F F F F F F F F F F F F F F F F F F F RI QFQ petrl hrteristis nd tmospheri e'etsF F F F F F F F F F F F F RQ QFR he erth oservtion systemF F F F F F F F F F F F F F F F F F F F F F F RW QFS qrphil onsequene of the onditionl independeneF F F F F F F F SH QFT qrphil onsequene of summing or mximizing out the vriles in eF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SH QFU ixmples of tringulted grphsF F F F F F F F F F F F F F F F F F F F F SQ QFV hidden wrkov hin F F F F F F F F F F F F F F F F F F F F F F F F F F F SQ RFI smges re orgnized ording to their resolutions in pyrmid strutureF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F TH RFP petrl hrteristis nd tmospheri e'etsF F F F F F F F F F F F F TI RFQ petrl hrteristis nd tmospheri e'etsF F F F F F F F F F F F F TT RFR floky rtifts using the method proposed in vferté et lFD PHHH using the ww riterionF F F F F F F F F F F F F F F F F F F F F F F F F TV RFS xeighorhood system in uto et lFD IWWTF F F F F F F F F F F F F F F TW RFT wo hierrhil struturesX @A qudEtree with three levelsY @A trunE ted tree with two levelsF F F F F F F F F F F F F F F F F F F F F F F F UH RFU ww estimtion on the qudEtree s proposed in oisinD PHIPF F F F UI SFI wultiEtemporl hierrhil strutureF F F F F F F F F F F F F F F F F F US SFP wultiEdte ww estimtion on the qudEtreeX aP nd two dtesF F VI SFQ ixmple of hp modeling using iw lgorithmF F F F F F F F F F F F VQ SFR ixmples of léides time series F F F F F F F F F F F F F F F F F F F F VT SFS ixmples of qeoiye time seriesF F F F F F F F F F F F F F F F F F F F F VT SFT qround truth for the léides imge quired in PHIQF F F F F F F F F VU SFU everge of the overll uries otined on the test sets of ll indiE vidul dtes results using severl wvelet fmiliesF F F F F F F F F F F F VV SFV yverll ury of the proposed method on the test set of the léiE des dt set s funtion of the prmeters ϕ nd θF F F F F F F F F VW SFW hetils of the the lssi(tion mps otined y the proposed method when pplied to the léides dt set with di'erent vlues of ϕ nd θF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F VW SFIH @A floky rtifts otined using qudEtree struture vi ww forE multionD @A redution of these loky rtifts using the proposed methodF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F WP SFII glssi(tion mps otined from léides imgesD @ gxi distriE ution eirus hAF F F F F F F F F F F F F F F F F F F F F F F F F F F F F WR SFIP glssi(tion mps otined from qeoiye imgesF F F F F F F F F F F WS TFI ensitivity to loud over nd ojet size using di'erent wvelength rngeF F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IHH TFP wultivrite sttistil modelingF F F F F F F F F F F F F F F F F F F F F IHI TFQ he evolution of the proility density of the generlized qmm distriution s funtion of σ mn @AD ν mn @AD nd κ mn @AF F F F F F F IHQ TFR wultiEsensor hierrhil strutureF F F F F F F F F F F F F F F F F F F F IHR TFS @A e imge @ esAD @A one hnnel from the optil imge @ qeoiyeA F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IHT TFT esult using the (rst methodF F F F F F F F F F F F F F F F F F F F F F F IHU TFU wultiEsensor hierrhil struture @se PAF F F F F F F F F F F F F F F IIH TFV pirst site used for experimentsX @A léides @ gxi distriution eirus hAD @A gu @ esAD nd @A P imges @ geAF F F F F III TFW eond site used for experimentsX @A léides @ gxi distriution eirus hAD @A gu @ esAD nd @A P imges @ geAF F F F F IIP TFIH @A esult using only optil léides imgesD @A esult using oth léides nd gu quisitionsD @A esult using oth léides nd P quisitions @dA esult using ll sensorsF F F F F F F F F F F F F F IIQ TFII @A esult using only optil léides imgesD @A esult using oth léides nd gu quisitionsD @A esult using oth léides nd P quisitions @dA esult using ll sensorsF F F F F F F F F F F F F F IIR UFI egulr retngulr lttie

X of size m × n F F F F F F F F F F F F F F IIV UFP gusl neighorhoodF F F F F F F F F F F F F F F F F F F F F F F F F F F IIW UFQ ryrid struture F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IPI
UFR esult using the (rst methodF F F F F F F F F F F F F F F F F F F F F F F IPT UFS esult using the (rst methodF F F F F F F F F F F F F F F F F F F F F F F IPU irthqukesD tsunmisD )oodsD (res nd volni eruptions re mong the mny nturl dissters we re exposed to on irthF es well s eing dngerous to humnsD these events ply importnt roles in multiple soilD eonomiD nd humn viewpointsF ped with suh phenomenD there is ruil need for dequte nd rpid dmge ssessmentF roweverD when n extensive re is 'etedD the ssessment of the dmge eomes omplex nd time onsuming tskD usully ondued y humn expertsF xonethelessD the pilities to monitor the irth9s surfe using omputer vision lgorithmsD notly in urn nd uiltEup resD for environmentl dissters nd to ssess the ground e'ets nd dmge of suh events ply importnt rolesF qiven monitored reD remote sensing llows elements t risk to e identi(ed nd their vulnerilities to e qunti(edF efter disster tully oursD this prior extrted informtion n e omined with results of multiEtemporl remote sensing imge nlysisD nillry informtionD nd possile physillyEsed modeling to estimte dmge erpio et lFD PHIPF sn this frmeworkD urte nd timeEe0ient lssi(tion methods for singleE dte nd multiEtemporl imgery re importnt tools required to support rpid nd relile extrtion of informtion on monitored regionD espeilly when n extenE sive re is onsideredF qiven the sustntil mount nd vriety of dt ville urrently from lstEgenertion veryEhigh resolution @rA stellite missionsD the min di0ulty is to develop lssi(er tht utilizes the ene(ts of multiEresolutionD multiEdteD multiEsensorD nd multiEfrequeny input imgeryF prom n pplitionE oriented viewpointD the gol is to tke dvntge of this vriety of input soures to mximize ury nd e'etiveness of the resulting themti mpping produtsF prom methodologil viewpointD this gol lims for the development of novel dt fusion tehniques tht re )exile enough to support the joint lssi(tion of imE ges olleted on the sme re y di'erent sensors t di'erent times nd ssoited with multiple sptil resolutions nd wvelength rngesF sn this mnusriptD this fuE sion prolem is ddressed y developing four novel tehniquesD whose ommon trit is mthemtil formliztion sed on hierrhil wrkov rndom (eld @wpA modelsF yn the one hndD the use of multiEresolution nd multiEnd imgery hs een previously shown to optimize the lssi(tion results in terms of ury nd omputtion timeD nd on the other hndD the integrtion of the temporl dimension into lssi(tion sheme n oth enhne the results in terms of reliility nd pture the evolution in time of the monitored reF roweverD the joint prolem of the fusion of severl distint dt modlities @eFgFD oth multiEresolution nd multiE temporl or multiEsensorA hs een muh more srely ddressed in the remote sensing literture so frF he next prgrphs rie)y summrize the key ides of the developed methods nd how their presenttions re orgnized within the vrious hpters of the thesisF Addressed problems and thesis organization sn this mnusriptD set of hierrhil methods to fuse multiEdteD multiEresolutionD multiEsensorD nd multiEfrequeny remote sensing imgery for supervised lssi(E tion is developed nd experimentlly vlidted with hllenging multiEmodl imE gery from the topil test site of ritiF Chapter 1 his hpter presents n introdution to remote sensing imges nd the di'erent sensors nd their propertiesD with speil fous on optil nd syntheti perture rdr @eA dtX how do the sensors workD wht re the possile sensor vrintsD wht re their dvntgesD weknessesD usesD nd wht properties of given sene re oserved with themD re mong the items relled in this hpterF he motivtion to jointly use heterogeneous sensors is lso disussedF Chapter 2 his hpter summrizes vrious pprohes to dt fusion proposed in the literE ture with regrd to multiEmodl remote sensing imgery for lssi(tion purposesF pirstD vriety of multiEsle nd multiEresolution fusion pprohes re reviewedF henD the omplementrity etween vrious sensorsD espeilly when operting in the mirowve nd optil rngesD is stressed y presenting severl models used in the litertureF pinllyD thorough review of multiEtemporl remote sensing imge nlysis methods is presentedF Chapter 3 his hpter provides ll neessry kground out wpsD suh s proilisE ti imge models nd methodologil tools for lssi(tionD nd gives fyesin justi(tion for the form of their energy funtionsF sn dditionD the grphil repE resenttion of wps is detiled in order to point out n importnt lss of wp models tht rely on proilisti uslity onept ptured y the ftoriztion of the prior distriution in term of usl trnsitionsF Chapter 4 he ojetive of this hpter is to desrie multiEresolution lssi(tion tehniques sed on hierrhil wps using qudEtree struturesF he use of qudEtrees to model sleEtoEsle intertions is justi(ed y their uslity properties over sle nd y the possiility of employing fst optimiztion methodF roweverD these strutures my indue loky rtifts in the (nl lssi(tion mpF his hpter lso presents severl tehniques to mitigte suh undesirle e'etF Chapter 5 huring this hpterD novel frmework for joint multiEtemporl nd multiEresolution lssi(tion is presentedF wultiEdte nd multiEresolution imge lssi(tion is sed on expliit sttistil modeling through hierrhil wpF he proposed model llows oth the input dt olleted t multiple resolutions nd dditionl multiEsle fetures derived through wvelets to e fusedF he desried pproh onsists of supervised fyesin lssi(er tht omines @iA joint lssEonditionl sttistil model for pixelEwise informtion nd @iiA hierrhil wp for sptioE temporl nd multiEresolution ontextul informtionF tep @iA ddresses the modE eling of the sttistis of the spetrl hnnels quired t eh resolution nd onE ditioned to eh lssF tep @iiA onsists of integrting this sttistil modeling in hierrhil wrkov rndom (eld for eh dteF e novel element of the proposed pproh is the use of multiple qudEtrees in sdeD eh one eing ssoited with eh new ville imge t di'erent dtesD to hrterize the temporl orE reltions ssoited with distint imges in the input time seriesF he trnsition proilities etween the sles nd etween di'erent dtes determine the hierrE hil wp euse they formlize the uslity of the sttistil intertions inE volvedF snferene nd proility density modeling re omplished y integrting the mrginl posterior mode @wwA riterionD the modi(ed wetropolis dynmis @wwhA energy minimiztion tehniqueD (nite qussin mixture modelsD nd the stohsti expettionEmximiztion @iwA lgorithmF ixperimentl results using multiEtemporl nd multiEresolution léides imges on riti y employing this new hierrhil model for time series lssi(tion re presentedF Chapter 6 sn this hpterD two methods re proposed for the joint supervised lssi(tion of multiEsensor imges inluding e nd optil omponents nd quired t multiple sptil resolutionsF he rtionle of oth pprohes is to tke ene(t of the dt fusion pilities of hierrhil wps while the omputtion of the joint sttistis of optil nd e dtD for whih no losed form prmetri models re villeD is voidedF e novel pprohD sed on multiple qudEtrees in sdeD to multiE sensor nd multiEresolution fusion is desriedF sn this frmeworkD the (rst proposed method ddresses the generl prolem of the joint lssi(tion of e nd n opE til imge quired over the sme re t high sptil resolutionsF por eh sensorD one of these two input imges is ssoited with seprte qudEtree struture on the sis of its resolutionF he proposed pproh formlizes supervised fyesin lssi(er within this multiple qudEtree topology tht omines lssEonditionl sttistil model for pixelEwise informtion nd hierrhil wp for multiEsensor nd multiEresolution ontextul informtionF he seond proposed method fouses on the spei( se of the fusion of multiEfrequeny e dt olleted y the gywyEkywed @ ndA nd eheeEP @g ndA sensors together with opE til léides dtF e multiple qudEtree struture is used gin ut optil nd e dt re jointly inluded in eh qudEtree to tke ene(t of oth the est resolution ville from eh sensor nd the synergy etween the dt o'ered y these opil speEorne missions for irth oservtionF Chapter 7 hroughout this hpterD the qudEtree pproh is extended nd novel ontextul multiEsle tehnique is proposed in order to lssify multiEresolution remote sensE ing dtD inorporte sptil ontextul informtionD nd mitigte possile loky rtifts while keeping the uslity of the hierrhil modelF his is omplished y proposing novel integrted tehnique tht omines the hierrhil wp nd wrkov mesh pprohes to tke ene(t of their uslity properties in the sle nd sptil dominsD respetivelyF he fous is on the fusion of multiEresolution nd sptilEontextul informtion for the supervised lssi(tion of singleEdte imgeryF e novel formultion of the ww riterion is developed nd preliminry experimentl nlysis is ondutedF

Conclusion and perspectives

sn this hpterD onlusions on the proposed methods re drwnD long with omE ments on their possile relevne in the urrent remote sensing frmeworkD nd future possile extensions re rie)y outlinedF 1.1 Remote sensing system emote sensing is the disipline whose gol is to otin informtion out n oE jetD re or phenomenon through the nlysis of dt quired y devie whih is not in physil ontt with the ojet eing investigted villesnd et lFD PHIRF sn modern usge nd in the pplition to irth oservtion @iyA the term generlly refers to the use of eril or stellite sensor tehnologiesD using vrious tegories of sensorsY we remotely ollet dt tht my e nlyzed to otin informtion out the ojet under investigtionF he remotely olleted dt n e of mny forms tht re potentilly ville from the energy (led rising from the irth9s surfe inluding spectralD spatial nd temporal (elds vndgreeD PHHSF he vritions of these (elds implies vritions in fore distriutions @for exmple mesuring qrvity from moving irrftAD ousti wve distriutions @for instne using torque nd tire pressure sensors in utomotive pplitionsA or eletromgneti energy distriuE tions @for exmple vi rdr sensors onEord rti(il stellitesAF hough qrvity nd ousti (elds re useful for this purpose within mny pplitionsD we will fous on the eletromgneti energy sensors tht re urrently eing operted from irorne nd speEorne pltforms to gther informtion remotely from the irth surfeF pigure IFI shemtilly illustrtes the generlized proesses nd elements involved in irth oservtion systemF es the (gure showsD the system my e divided into three si prtsX the seneD the sensorD nd the proessing systemF

The scene is the prt of the system to e nlyzedD it lies in front of the sensor nd onsists of the irth9s surfe nd its tmosphereF his portion of the system is hrterized y not eing under humn ontrol during ll the oservtion proessF

The sensor is the prt of the system tht ollets the dt to e nlyzedF here re minly two sensor tegories X Passive sensors whih mesure re)eted sunlight tht ws emitted from the sun nd active sensors whih hve their own soure of 4illumintion4 nd mesure re)eted @kstteredA energyF purthermoreD sensors my e rried on either n airborne @irrftD droneA or space-borne @stelliteA pltforms depending on the use of the prospetive imgesF he sensor system is usully under humn ontrol espeilly during the system design phseF

The processing system is the prt of the system ontrol whose gol is to extrt informtion from ontrol the dt provided y the sensorF he pility of urrent remote sensors to generte dt fr exeeds the urrent pity to hndle these dtF roessing dt provided y sensors into n interpretle formt n e n e'ort entiling onsiderle thoughtD instrumenttionD timeD experieneD nd referene dtF rumn intervention in dt proessing is nd will mostly ontinue to e essentil to the produtive pplition of remote sensing informtionF his proessing n e done prtly on ord nd prtly in ground sttionsF sn the reminder of this hpterD the si priniples underlying the remote sensing proess re disussedF he fundmentls of eletromgneti energy re onsidered y going through the intertion of the energy with the tmosphere nd with irth surfeF hese sis will permit to oneptulize the min tehnologies in remote sensing systems ording to the eletromgneti wvelength rnge eing usedF 1.2 Energy sources and radiation principles isile light is only one of mny forms of eletromgneti energyF yther fmilir forms re invisile eletromgneti rditionsD suh s rdio wvesD ultrviolet rys nd ErysF ell this energy follows the si wve theoryF es shown in pigure IFP this theory desries eletromgneti energy s trveling wve through spe in rippling eletri nd mgneti (eld t the veloity of lightD cF the distne from one wve pek to the next one is the wvelength λ nd the numer of peks pssing (xed point in spe per unit time is the wve frequeny νF he mplitude of n eletromgneti wve is the height of the wve rest ove the undistured positionF pigure IFPX en eletromgneti wveF sn remote sensing it is most ommon to hrterize eletromgneti wves y their wvelength lotion within the eletromgneti spetrum @pigure IFQA whih rnges from the shorter wvelengths @inluding qmm nd XErysA to the longer wvelengths @inluding mirowves nd rodst rdio wvesAF here re severl regions of the eletromgneti spetrum whih re useful for remote sensing euse (rstD they n e relted to irth oservtion pplitions nd seondD the tmoE sphere is reltively trnsprent in those rnges @see elowAF he visile portion of the plot in pigure IFQ is n extremely smll oneD sine the spetrl sensitivity of the humn eye extends only from pproximtely 0.4 µm to 0.7 µmF here re lot of rditions round us whih re 4invisile4 to our eyesD ut n e deteted y other remote sensing instruments nd used to our dvntgeFhe portion of the spetrum of more reent interest to remote sensing is the mirowve region from out I mm to I mF his overs the longest wvelengths tht re ommonly used for remote sensingF he use of suh rnge is not ritrryF sn ft some wvelengths re inessile s result of intertion of the wve with the tmosphere nd irth surfe feturesF pigure IFQX he eletromgneti spetumF 1.3 Interactions with the atmosphere srrespetive of its soureD the rditions used for remote sensing @iFeFD pssive sE xs @rdition from the sunA nd tive rdr @rdition from the sensorAA hve to trvel some distne through the tmosphere efore they reh the irth9s surfeF feuse of their vrietyD prtiles nd gses in the tmosphere n 'et the inomE ing light nd rditionF hese e'ets re priniplly used through the mehnisms of tmospheri scattering nd absorption villesnd et lFD PHIRF Scattering ttering is the @deterministilly unpreditleA di'usion of the eletromgE neti rdition used y the intertion with prtiles in the tmosphereF eording to the size of prtiles in intertionD there re three types of stE teringF IF Rayleigh scattering is ommon when rdition interts with tmospheri moleules nd other tiny prtiles tht re very smll in dimeter omE pred to the wvelength of the interting rditionF yleigh stter is one of the primry uses of 4hze4 in imgeryF PF Mie scattering ours when tmospheri prtile dimeters re just out the sme size s the wvelength of the rdition eing sensedF ter vpor nd dust re ommon uses of wie sttering whih tends to in)uene longer wvelengths thn those 'eted y yleigh stteringF QF Non-selective scattering is the most othersome phenomenon espeilly for optil sensorsF his ours when the dimeters of prtiles using stter re muh lrger thn the wvelength of the energy eing sensedF ter droplets nd lrge dust prtiles n use this type of stteringF hey ommonly hve dimeter in the rnge 5 to 100 µm nd stter ll visile wvelengthF xonEseletive sttering gets its nme from the ft tht equl quntities of lueD greenD nd red lights re stteredD hene fog nd louds pper white in optil imgesF

(a) (b) (c)
pigure IFRX petrl hrteristis of @A nominl lkEody energy souresD @A tmospheri e'etsD nd @A sensing systemsF Absorption sn ontrst to stteringD this phenomenon uses moleules in the tmosphere to sor energy t vrious wvelengthsF he most e0ient sorers of solr rdition in this regrd re wter vporD ron dioxideD nd ozoneF feuse these gses tend to sor eletromgneti energy in spei( wvelength ndsD they strongly in)uene 4where we look4 spetrlly with ny given remote sensE ing systemF he wvelength rnges in whih the tmosphere is prtiulrly trnsmissive of energy re referred to s atmospheric windowsF pigure IFR @A shows the two most ommon energyGrdition soures @the sun nd the irthAF sn pigure IFR @AD spetrl regions in whih the tmosphere loks energy re shdedF eordinglyD only nonEloked spetrl regions ould e used for reE mote sensingF he importnt point to note from pigure IFR is the interdepenE dene etween the primry soures of eletromgneti energyD the tmospheri windows through whih soure energy my e trnsmitted to nd from surE fe feturesD nd the spetrl sensitivity of the sensors ville to detet nd reord the energyF 1.4 Energy interactions with Earth surface features dition tht is not sored or sttered in the tmosphere n reh nd intert with the irth9s surfeD s shown in pigure IFSF here re three fundmentl energy intertions tht n tke ple when energy is inident @sA upon the surfeF rious frtions of the energy re re)eted @AD sored @eAD ndGor trnsmitted @AF he totl inident energy will intert with the surfe in one or more of these three wysF he proportions of eh will depend on the wvelength of the energy nd the mteril nd ondition of the fetureF pigure IFSX fsi intertions etween eletromgneti energy nd irth surfe fetureF fy mesuring the energy tht is re)eted @or emittedA y trgets on the irth9s surfe over vriety of di'erent wvelengthsD we n uild up spetrl response for tht ojetF fy ompring the response ptterns of di'erent fetures we my e le to distinguish etween themD where we might not e le toD if we only ompred them t one wvelengthF por exmpleD wter nd vegettion my re)et somewht similrly in the visile wvelengths ut re lmost lwys seprle in the nerE infrred @see pigure IFTAF unowing where to 4look4 spetrlly nd understnding the ftors whih in)uene the spetrl response of the fetures of interest re ritil to orretly interpret the intertion of eletromgneti rdition with the surfeF pigure IFTX petrl re)etne of vegettion nd wterF 1.5 Temporal domain reretoforeD ll these intertion @in the tmosphere or with the irth surfe feE turesA re mde in (xed timeF hespite our tendeny to onsider irth s sttiD it is tully dynmi nd everEhnging plnetF petrl response n e quite vrileD even for the sme trget typeD nd n lso vry with time @eFgF 4greenE ness4 of levesD see pigure IFUA nd lotionF herefore the temporl domin ply primry role in remote sensing nlysis pigure IFUX petrl re)etne of ok levesF 1.6 Satellites and sensors properties 1.6.1 Characteristics of remote sensing images fefore we go further in the remining of this hpterD whih fouses in more deE til on sensors nd their hrteristisD we need to de(ne nd understnd few fundmentl terms nd onepts ssoited with remote sensing imgesF Spatial Resolution, Pixel Size, and Scale he detil disernile in n imge is dependent on the spatial resolution of the sensor nd refers to the size of the smllest possile sptil feture tht n e detetedF sndeedD remote sensing imge ould e represented nd displyed in digitl formt y sudividing the imge into smll equlEsized nd shped resD lled piture elements or pixelsD nd representing the rightness of eh re with numeri vlue quntized on (nite set of levelsF st is importnt to distinguish etween pixel size nd spatial resolution E they re not interE hngeleF sn ftD sf sensor hs sptil resolution of Im nd n imge from tht sensor is displyed t full resolution with idelly no lurringD eh pixel represents n re of 1m × 1m on the groundF sn this se the pixel size nd resolution re the smeF roweverD it is possile to disply n imge with pixel size di'erent thn the resolutionF wore generllyD imge quisition introdues @possily smll ut nonzeroA lurring due to the point spred funtion of the sensorF smges where only lrge fetures re visile re sid to hve coarse or low resolutionF sn ne or high resolution imgesD smll ojets n e detetedF Spectral Resolution es shown in the eginning of the urrent setionD di'erent lsses of fetures nd detils in n imge tken y pssive sensor n often e distinguished on the sis of their responses over distint wvelength rngesF husD one would require sensor whih ould distinguish rod wvelength rngesF et this pointD spectral resolution desries the ility of sensor to de(ne (ne wveE length intervlsF he (ner the spetrl resolutionD the nrrower the wvelength rnge for prtiulr hnnel or ndF Radiometric resolution hile the rrngement of pixels desries the sptil struture of n imgeD the radiometric resolution desrie the preision with whih the mesurement tken on eh pixel of n imge is desriedF st is usully expressed in terms of the numer of quntiztion levels or the orresponding numer of its used to ode these quntiztion levels with inry formtF husD the di'erene in the level of detil disernile depends on the numer of its used in representing the energy reordedF Temporal Resolution he onept of temporal resolution is importnt in speEorne remote sensE ing systemD sine remote sensing stellite generlly reEvisit the sme imged re fter one entire orit yleF sndeedD the solute temporal resolution of remote sensing system is expressed in terms of the period of time needed to reEimge the ext sme re t the sme viewing ngleF gontemporry high resolution stellite missions of irth osefrvtion usully provide pointing funtionlityD iFeFD the viewed re n e pointed @to some extentA on sis of userGopertor9s hoieF sn this seD temporl resolution nd revisit time eome nonEperiodi nd generlly irregulrF 1.6.2 Passive vs. Active Sensing es mentioned in setion IFID remote sensing systems whih mesure energy tht is nturlly ville from the sun re lled passive sensorsF ssive sensors n only e used to detet energy when the nturlly ourring energy is villeF por ll re)eted energyD this n only tke ple during the time when the sun is illuminting the irthF Active sensorsD on the other hndD provide their own energy soure for illumintionF he sensor emits rdition whih is direted towrd the trget to e investigtedF he rdition re)eted from tht trget is deteted nd mesured y the sensorF porthwithD n inresing mount nd vriety of lstE genertion veryEhighEresolution stellite missions re villeF pigure IFV shows the rpid growth of the numer of oservtion systems operting in oth optil nd mirowve spetrum s well s their performne in term of resolutionsF 1.6.3 Passive sensors operating in the optical spectrum his setion fouses on stellite systems tht operte within the optil spetrumF yptil remote sensing mkes use of visileD ner infrred nd shortEwve infrred sensors to form imges of the irth9s surfe y deteting the solr rdition reE )eted from trgets on the groundF hi'erent mterils re)et nd sor di'erently t di'erent wvelengthsF husD the trgets n e di'erentited y their spetrl re)etne signtures in the remotely sensed imgesF yptil remote sensing sysE tems re lssi(ed into the following typesD depending on the numer of spetrl nds used in the imging proessF Panchromatic imaging system X he sensor is single hnnel detetor sensiE tive to rdition within rod wvelength rnge usully enompssing the visile rnge nd possily prt of the nerEinfrred rngeF Multi-spectral imaging system X he sensor is multihnnel detetor with few spetrl ndsF ih hnnel is sensitive to rdition within reltively rod wvelength ndF he resulting imge is multilyer imgeF Hyper-spectral imaging system X e hyperEspetrl imging system is lso known s n 4imging spetrometer4F it quires imges in out hundred or more very nrrow spetrl ndsF (a) Systems operating in optical spectrum (b) Systems operating in microwave spectrum pigure IFVX he growth of the numer of oservtion systems operting in oth optil nd mirowve spetrum http://rsde.fbk.eu/ es shown in pigure IFVD irth oservtion dt re eoming ville t inE resingly (ner resolutionsF ery righ esolution @rA stellite imgery lredy in existene or due to e lunhed in the ner future o'ers suEmeter resolutionF sn the remining of this setion we present the min hrteristis of the optil sensors whose imges were used during the hFhF Pléiades he léides system ws designed under the prenhEstlin ypiy progrm @ypE til 8 dr pederted irth yservtionA etween PHHI nd PHHQF he léides onstelltion is omposed of two veryEhighEresolution optil stellites tht provide the overge of irth9s surfe with repet yle of PT dysF he pnhromti nd multispetrl dtsets re ville seprtely or n e purhsed s fused @pnEshrpenedA t HFSm of resolutionF he min hrteristis of the léides system re reported in le IFI GeoEye qeoiyeEI is very high resolution stellite tht ws lunhed in PHHV from nE denerg eirse nd is owned nd operted y higitlqloe sn @gyD eAF he stellite ollets imge dt down to pixel resolution of HFRI mF he pnhroE mti nd multispetrl dtsets re ville seprtely or n e purhsed s fused @pnEshrpenedA t HFRImF he min hrteristis of the qeoiye system re reported in le IFP 1.6.4 Active sensors operating in the microwave spectrum en inresing mount of vlule resoure informtion is eing quired y senE sors tht operte in the mirowve portion of the eletromgneti spetrumF etive imging ehe systems re generlly onsidered to inlude mirowves with wveE length from 1 mm to 1 m nd re the sujet of this setion @for more detils refer to renderson et lFD IWWVAF prom remote sensing stndpointD there re tree opE ertionl hrteristi tht distint mirowve rditionF IF etive sensors sed on mirowves re independent of the nturl soure of illumintionF hereforeD mirowve stellites quire imges during dytime s well t nightF PF fsed on the sttering phenomen detiled in setion IFQD mirowves n penetrte the tmosphere in ll wether onditionsD inluding hze light rinD snowD loud nd smokeD depending on the wvelengths involvedF QF here is no diret reltionship etween mirowves sttered ye the irth surfe to their ounterprts in the visile portion of the spetrumF por inE stneD surfe tht pper 4rough4 in the visile portion of the spetrum my e 4smooth4 s seen y mirowvesF

Spectral Bands

• nX HFRUEHFVQ µmY

• flue a HFRQEHFSS µmD

• qreen a HFSHEHFTP µmD

• ed a HFSWEHFUI µmD

• xer snfrred @xsA a HFUREHFWR µm Ground Sampling Distance es further omprisonD the visile prt of the eletromgneti spetrum n e sid to inlude the redD greenD nd lue spetrl regionsF imilrlyD the tive mirowves region inludes XD CD L nd K mong othersD tht refer ti spei( segE ments of the mirowve portion of the eletromgneti spetrumF o illustrteD n X nd system would e rdr tht opertes t single wvelength within this nd @eFgFD 3.2 mFAF e spei( ehe tehnique is used for remote sensingX ynE theti eperture dr @eA imgingD in whih mirowve pulses re trnsmitted y n ntenn towrds the irth surfeF he mirowve energy sttered k to the sperft is mesuredF he e mkes use of the rdr priniple to form n imge with eptly (ne sptil resolution y utilizing the motion of the stellite on the oritF sn the following of this setion the sis of e re relled @for more detiled explntion of e the reder ould refer to ermin nd predD PHHRAF e e system produes twoEdimensionl imgeF yne dimension in the imge is lled range nd is relted to of the distne from the rdr to the trgetF he range is determined y mesuring the time from the trnsmission of pulse to the reeption of the orresponding eho from trgetF he rnge resolution is deterE mined y the trnsmitted pulse width nd wveformD @iFeFD nrrow pulses yield (ne rnge resolutionA nd y suitly (ltering to eho signl @pulse ompressionAF he seond dimension is lled azimuth nd is perpendiulr to the rnge diretionF o otin (ne azimuth resolutionD physilly lrge ntenn would e needed to fous the trnsmitted nd reeived energy into shrp emF he shrpness of the em de(nes the azimuth resolutionF he e tehniques exploit the theory of ntenn rrys nd of the hoppler e'et to improve zimuth resolutionF hen disussing mirowve energyD the polriztion of the rdition is lso imporE tntF olriztion refers to the orienttion of the eletri (eld with respet to n orthonorml sis perpendiulr to the diretion of irrditionF wost rdr senE sors re designed to trnsmit mirowve rdition either horizontlly polrized @rA or vertilly polrized @AF imilrlyD the ntenn reeives either the horizontlly or vertilly polrized omponent of the ksttered (eldD nd some rdrs n reeive othF COSMO-SkyMed gywyEkywed is n stlin stellite onstelltion ommissioned nd operted y the stlin spe geny @esAF st onsists of R mediumEsize stellitesD eh one equipped with mirowve highEresolution syntheti perture rdr @eA operE ting in End @wvelength of QFP mAD hving THH km single side ess ground reD oriting in sunEsynhronous orit t TPHkm height over the irth surfeD with the pility to hnge ttitude in order to quire imges t oth right nd left sides of the stellite ground trkF es shown in pigure IFW nd le IFQ severl omintions etween imge size nd sptil resolution hve een hosenX • e potlight modeD for 1m resolution over smll imgesY RADARSAT-2 drstEP is gndn ommeril irth oservtion stellite operted y the gndin pe egeny @geAF st utilizes syntheti perture rdr @eA senE sor to imge the irth t single mirowve wvelengthD in the gEnd @wveE length of SFT mAF pigure IFIH gives visul representtion of the overge re of eheeEP imging mode long the suEstellite ground trkD s well s the stellite veloity vetorF he em modes re shown in order of deresing overge reX

• vow esolution wode @IHHmAY • wedium esolution wode nd vow xoise G hip hetetion wode @SHmAY • wedium esolution wode @QHmAY • wedium esolution wode nd udEol wode @ITmAY • righ esolution wode @SmAY • ery righ esolution wode @QmAY • potlight wode @ImAF pigure IFIHX he quisition modes of drstEP tellite @ geA 1.7 Remote sensing applications tellite informtion is fundmentlly importnt to solve some of the mjor hlE lenges of our timeF por issues rnging from defenseD wether preditionD nd minerl explortion to pplitions s diverse s ivil protetionD risk mngementD urn mppingD preision griultureD ommeril (shing nd infrstruture mngementF emote sensing provides welth of informtion t glol sleF eordinglyD n informed deision n e mde y involving mny proessing tsks inludingX Otho-rectication rp n imge to its lotion on the irthF Image Classication gtegoriztion of pixels sed on re)etne into di'erent lnd over lssesF Spectral Analysis y employing smll numer of refully hosen spetrl nds spred ross the visile nd infrred regions of the eletromgneti spetrumF por exmpleD using nonEvisile prts of the eletromgneti spetrum to deE termine if forest is helthyF Multi

Data fusion in Remote sensing

he literture in dt fusion is extensiveD inditing the intense interest in this topi @highlighted y the shrp inrese in the numer of ppers pulished in the mjor remote sensing journlsD nd the inresing numer of relted sessions in internE tionl onferenesAF sn ftD dt fusion gve rise to ontinuing trdition in remote sensingD sine irth oservtion is y de(nition dynmi @thus implying the multiE temporl pility of remote sensing sensorsAD multiEresolution @multiple sptil nd spetrl resolutionsA nd relted to di'erent physil quntities @thus requiring multiEviewGmultiEsensor pilityA glrk nd uilleD PHIQD ltz et lFD IWWHF ht fusion is de(ned di'erently depending on the (nl gol of the userF sn ftD ohl nd vn qenderenD PHIRD vi et lFD IWWS onsidered dt fusion in remote sensing s the omintion of two some lgorithmsF his my inludeD multi-resolution fusion and pansharpening techniques with the im is to otin multiEspetrl imges of inresed sptil resolution ivone et lFD PHISD resolution blending tht onsists in providing time series of dt t their mximum sptil nd spetrl resolutions @referred to s prllel pnshrpening in multiEtemporl dominA rung nd ongD PHIP nd data fusion for missing information reconstruction y using omplementry dt ng nd vingD PHIRF en lterntive perspetive is to de(ne dt fusion in remote sensing s deision fusion tehnique whih omines the informtion tht is otined from di'erent dt sets nd provides su0ient generliztion pility ldD IWWWF eording to this de(nitionD ny type of imge proessing tht omines two or more dt setsD suh s imge lssi(tion or tmospheri orretion or pplition of vegettion indiesD should e onsidered dt fusionF ithin the former de(nitionD vrious proedures of dt fusion tehniques hve een proposed in the literture with regrd toD on the one hndD the application requirements suh s the vilility of ground referene dtD the olleted prior informtion ndGor some nillry dt tht n e used in the development of the system ording to multiEsoure proessing rhitetureF yn the other hndD it is importnt to properly understnd the user needs with respet to eonomi ostsD proessing time nd performneF pigure PFI summrizes the generl rhiteture of dt fusion tehniquesF 2.2 Multi-sensor and multi-resolution approaches 2.2.1 Multi-resolution approaches es exposed eforeD the vilility of remote sensing imgery of vrying resolution hs inresedF werging imges of di'ering sptil resolution hs eome signi(nt opertion in the (eld of digitl remote sensingF e vriety of di'erent multiEsle fuE sion pprohes hve een developed sine the lte VHsF sn the following we give n exerpt on the most ommon pprohes found in the litertureF e divided them into the two groupsD @iA trnsformtion tehniques nd @iiA modeling tehniquesF wethods in @iA onsist in repling the entire set of multiEsle inforE mtion y single omposite representtion whih inorportes ll releE vnt dtF he multisle trnsformtions usully employ yrmid rnsE forms furtD IWVRD the hisrete velet rnsform @hA iellD PHHQD porster et lFD PHHRD hng nd rongD PHHSD the ndeimted velet rnsform @A okingerD IWWTD ghini nd rouineD PHHQD the hulEree gomplex velet rnsform @hgA hemirel nd enrjfriD PHIHD sql et lFD PHIQD the gurvelet rnsform @gA xenini et lFD PHHUD ghoi et lFD PHHSD the gonE tourlet rnsform @gonAevijily et lFD PHHVD hh et lFD PHHV nd the xonsuE smpled gontourlet rnsform @xgA ng et lFD PHHUF @iiA inludes multiEsle pprohes with fous on the use of orser resolution of the dt set in order to otin fst omputtionl lgorithmsF sn seminl ppers fsseville et lFD IWWPD fsseville et lFD IWWP the sis for multiEsle utoreE gressive modeling in dydi trees ws introduedF ineD strightforwrd pprohes were performed to del with multiEresolution imges using trees érezD IWWQD utoD IWWSD vferté et lFD PHHHD ghrdinD PHHHD oisinD PHIPD redhli et lFD PHIRF hue to their importneD the susequent hpters provide more detiled view on the use of multiEsle modeling on treesF roweverD detiled review of some of these methods ould e found in qr0gne et lFD IWWSD illskyD PHHPF 2.2.2 Multi-sensor approaches wultiEsensor nlysis is proess deling with dt nd informtion from mulE tiple sensors to hieve re(nedGimproved informtion s ompred to the result tht ould e otined y using dt from only one individul soure @see rll nd vlinsD PHHID ltz et lFD IWWHD ohlD IWWVAF he lssi(tion ury of remote sensing imgesD for instneD is improved when multiple soure imge dt re introdued to the proessing @eFgFD xguyen et lFD PHIID qm et lFD PHIID housset nd qourmelonD PHHQD redhli et lFD PHISAF smges from mirowve nd optil sensors provide omplementry informtion tht helps in disriminting the di'erent lssesF everl proedures hve een introdued in the literture inludE ingD on one hndD postElssi(tion tehniques in whihD (rstD the two dt sets re seprtely segmentedD nd then the joint lssi(tion is produed y usingD for exE mpleD rndom forest @pA @eFgFD ske nd vn der vindenD PHHVAD support vetor mhine@wA with dEho kernels wuñozEwrí et lFD PHIHD rti(il neurl netE works @exxA ws nd ploresD PHHVF yn the other hndD other methods diretly lssify the omined multiEsensor dt y using for instneD sttistil mixture models @eFgFD housset nd qourmelonD PHHQD oisin et lFD PHIPD rendesD PHISAD entropy sed tehniques @eFgFD oerts et lFD PHHVAD fuzzy nlysis @eFgFD fenzD IWWWD troppin et lFD PHISAF purther moreD for omplex dtD espeilly when deling with urn resD rdr imges n ontriute to di'erent re)etne due to di'erenes in surfe roughnessD shpe nd moisture ontent of the oserved ground over@eFgFD frunner et lFD PHIHAF he use of multiEsensor dt in imge lssi(tion eomes more nd more populr with the inresed vilility of sophistited softwre nd hrdwre filities to hndle the inresing volumes of dtF he deision on whih tehnique is the most suitle is very muh driven y the pplition nd the typology of input remote sensing dtF pigure PFPX wultiEtemporl method tegoriztionF 2.3 Multi-temporal analysis approaches es disussed in ghpter ID stellite remote sensors for irth oservtion hve the pility to visit the sme geogrphi re severl timesF everl surE vey ppers provide thorough reviews of multiEtemporl remote sensing imge nlysis methodsD inghD IWVW lssi(es these pprohes into two tegoriesX @IA omprtive nlysis of independently produed lssi(tions for di'erent dtes tohnson nd usishkeD IWWVD zirnyi nd hdydehD PHIR nd @PA simulE tneous nlysis of multiEtemporl dt fzi et lFD PHHSD redhli et lFD PHIRD ingh et lFD PHIRF heresD wlilD IWVH points out two other si pprohes for multiEtemporl nlysisX @IA lssi(tion pprohes roerg et lFD PHISD redhli et lFD PHIRD welgni nd erpioD PHHQ nd @PA hnge mesurement @strti(tionA fenedek et lFD PHISD qm et lFD PHHTF etitjen et lFD PHII slightly hnges the fous y entering the de(nitions more on tempoE rl sle nd rings out three min usges of the time dimensionX @IA time s identi(erX time is only used to identify the informtionD iFeFD there is no ordering etween the imges of the series welgni nd erpioD PHHQD roerg et lFD PHISY @PA pirwise time orderingX time is used to struture the imgesD pirwise viu et lFD PHIPD fovolo nd fruzzoneD PHHVY @QA time ordering the sequeneX time is used to struture the imge series etitjen et lFD PHIID eer et lFD PHIPD redhli et lFD PHIRD redhli et lFD PHIRF yther sientists xelsonD IWVQD wilneD IWVV nd goppin et lFD PHHR hve employed more sophisE tite multiElss shemesF sn the following of this setionD the tegoriztion in fovolo nd fruzzoneD PHIS is doptedF rerefterD we present the most often used methods in the literture tht del with multiEtemporl dt y lssifying them into two min tegories s shown in pigure PFP 2.3.1 Multi-temporal fusion at feature level the im of these methods is to generte new fetures @iFeFD hnge indies in iEtemporl imge pirsA tht highlight multiEtemporl informtionD nd then to extrt hnges ourred in the temporl series or to perform more sophistited nlysis suh s ssessingD monitoringD nd prediting the dynmis of nturl lnd oversF purther moreD fusion t feture level is hrterized y severl prtiulr ftors tht render ine'etive some of the multiEtemporl imge nlysis tehniques typilly used in other pplition dominsF xonethelessD preEproessing steps re mndtoryF es shown in the generl shem of dt preEproessing illustrted in pigure PFQ these steps my inludeX

• Radiometric CorrectionX wultiEtemporl imge series re generlly imE pted y di'erenes in light onditionsD sensor lirtionD nd ground moisE tureF hese lst items might e onsidered legitimte ground hngeD though di'erenes in light nd tmospheri onditions etween imgesD for instneD n e mitigted y pplying rdiometri lirtion to the temporl seriesF sn the literture there re two mjor lirtion pprohes depending on the prtiulr pplition onsidered nd on the spei( informtion villeF he (rst one is the Absolute Calibration whih onsists in trnsforming the quntized pixel intensities @digitl numersA into the orresponding ground reE )etne vlues y employing rdiometri trnsfer models uurum et lFD PHIID fisws et lFD PHIQD wontopoli et lFD PHIQ or regression lgorithms pplied to groundEre)etne mesurements olleted during the dt quisition phse renderson et lFD IWWVD qordon nd worelD PHIPD fernstein et lFD PHIPF he seond tegory widely used is the Relative Calibration whih diretly modi(es the histogrmsD so tht the sme gry levels vlues in imges n represent similr re)etne vlues qdllh et lFD PHHHD okol et lFD IWWW • Geometric Corrections & Image RegistrationX xonEperfet lignment @registrtion woiseA nd di'erent quisition onditions @view ngleD shdE owsD F F F A re ommon prolems within the onsidered imge time seriesF sn ftD it is not possile to otin perfet lignment etween multiEtemporl imgesF his is minly due to lol defets in the geometries of the imE ges whih result in very ritil soure of noise lled registration noise wes et lFD IWWUD ve woigne et lFD PHIIF • Image FilteringX his step is not mndtory nd depends on the spei( sensor nd on the qulity of the imgesF e wide vriety of (ltering tehE niques re used in remote sensing @see for instne howengerdtD PHHTAF sn the se of optil imges one wy to rete smoothed imge time series is to use multiEpoint smoothing funtionsD suh s moving verges or polyE nomil (lters@delhovenD PHIID vitzky nd qolyD IWTRD ghen et lFD PHHRF he tsk is more delite with e imges sine they re inherently fE feted y multiplitive spekle noiseF hereforeD mny dptive (lters for spekle redution hve een proposedF sing for instneD prost (lter whih models the sene re)etivity prost et lFD IWVPD uun pilter sed on the minimum menEsqure error @wwiA riterion uun et lFD IWVSD twoEdimensionl ulmn (lter stisfying usl eutoEegressive model ezimiEdjdi nd fnnourD IWWID qmm we (lter sed on fyesin nlysis of the imge sttistis where oth signl nd spekle noise folE low qmm distriution frldi nd rmigginiD IWWSD nd wvelet trnsform s powerful tool for reovering e imges from noisy dt pukud nd riroswD IWWWF pigure PFQX qenerl shem of dt preEproessingF henD fter the preEproessing opertions rod rnge of methods n e pplied in order to extrt temporl informtion t feture level tht depend essentilly on the type of the sensor usedF indeedD the noise model in optil imges quired y pssive sensors is usully onsidered dditive nd the nturl lsses tends to follow qussin distriution ihrdsD PHIQF he spekle in e mplitude nd intensity imgesD whih is generted within the sttering phenomenonD ts through multiplitive modelF le PFI nd le PFP summrize feture level methods using optil nd e imgesD respetivelyF

Multi-temporal fusion at decision level heision or interprettion level nlysis indues methods tht use vlueEdded dt

where the input imges re proessed individully for informtion extrtionF the otined informtion is then omined pplying deision rules to reinfore omE mon interprettionD resolve di'erenes nd provide etter understnding of the oserved dtF wost prior studies of multiEtemporl nlysis hve involved lssi(E tion tehniques whih tully omine the hnge detetion nd hnge identiE (tion spets of using informtion to updte resoure surveysF yther spets of lssi(tion re the requirements of trining nd dditionl ground truth for suE pervised or unsupervised lssi(tion of imgesF smge trnsformtionD vegettion indiesD dvned lssi(tion methods nd integrtion of di'erent dt soures re often used to improve lssi(tion resultsF le PFQ summrizes the methods using multiEtemporl informtion t the deision level sed on lssi(tion pprohesF sn the present mnusriptD we will fous on supervised lssi(tion methods for multiEsoure dtF hese methods hve the dvntge of eing pplile to two or wo fyesin pprohes n generlly e dopted for this purposeF he 9sde9 lssi(er @eFgFD winD IWUVD redhli et lFD PHIRA removes the oupling etween the spetrl nd temporl dimenE sions nd lssi(es eh imge in the inE put series on the sis of itself nd of the previous imgesF he 9mutul9 pE proh lssi(es eh imge on the sis of the previous nd the susequent imE ges in the seriesD @eFgFD roerg et lFD PHISD welgni nd erpioD PHHQAF he two pE prohes re silly omplementry in terms of ppliilityX online proessing n e fesile within the sde dominD wheres the mutul domin n e generlly more pproprite for th proessingF erti(il neurl networks @exxA exx is nonprmetri supervised method nd hs the ility to estimte the properE ties of dt sed on the trining smplesF e kEpropgtion lgorithm is often used to trin the wultiEvyer ereptron @wvA neuE rl network model @eFgFD willer et lFD IWWSD fruzzone et lFD IWWWAF yther hnge multiEtemporl tehE niques snlude sptil sttistisEsed methodD fioE physil prmeter methodD ryrid hnge detetionD nsupervised methodsD ht minE ing pprohesD multiEresolution tehniquesF le PFQX wultiEtemporl tehniques t the deision level imgesF more imges tken y single or multiple sensors t the sme or t di'erent sptil resolutionsF he ville imges re temporlly nd sptilly orreltedF sn ftD temporl nd sptil ontextul onstrints re unvoidle in multiEtemporl dt interprettionF ithin this frmeworkD wrkovin wodels provide onvenient nd onsistent wy of modeling ontextEdependent sptioEtemporl entities originted from multiple informtion souresD suh s imges in multiEtemporlD multiEsensorD nd multiEresolution ontextF sn the next hpterD we introdue the sis of the ontextul lssi(tion using wrkovin modelsF Chapter 3

Markovian image models

Inverse problems and contextual information

ghnge detetionD imge lssi(tionD imge restortion re mny exmples tht show the numer nd diversity of inverse prolem in imge proessing xeelmniD PHHQF hese prolems n e regrded s the proess tht estimtes hidden informations @iFeFD ltent vrilesA x @eFgFD lnd over lss lelsA from observations y @eFgFD stellite dtA tthed to set of nodes @eFgFD pixelsA SF olutions to inverse prolems re sed on models tht link lels to the given oservtionF hese prolems re often ill-posed in the sense of rdmrd1 D due to the loss of inforE mtion tht ours when forming the oservtionsF o overome this illEposednessD mjor pproh is to promote ertin types of solutions y imposing prior knowledge on the lelsD whih my inludeD for instneD ontextul priors tht fvor sptil regulriztion of the onsidered signlsF o stress this spetD mny issues of imE ge nlysis n e modeled nd oped with y designing suitle energy funtions U (x, y) whih pture the intertions etween the unknown vriles x = {x s } s∈S to e estimtedD nd the oserved vriles y = {y s } s∈S D while inorporting the desired prior hrteriztionF 3.1.1 The Ising model snspired from ferromgnetismD the simplest theoretil desription of the intertion etween vriles is lled the Ising modelF yur strting point is lttieD whih is (nite set of regulrly sped points in spe of dimension d = 1, 2 or 3F sn dimension 1 we simply hve string of points on lineD whih n e enumerted from 1 to N F et eh point there is spin whih is either up or down t ny given on(gurtion ω = {ω i } 1≤i≤N s shown in pigure QFIF sn this ontext eh spin is funtionX

δ i (ω) = 1 if ω i is up -1 if ω i is down
en energy U is then ssigned to eh on(gurtionX

U (ω) = -α i δ i (ω) -β i,j δ i (ω)δ j (ω). @QFIA
he (rst term represents the in)uene of eh spin on the energyD regrdless of the other spinsF he seond term represents the energy ontriution used y the spin intertionsF sf β > 0D the intertion tend to keep neighoring spins in the sme diretion @attractive caseA otherwiseD neighoring spins with opposite orienttions re fvored @repulsive caseA risiD IWVVD fxterD PHHUF pigure QFIX yne dimensionl ssing modelF

The Potts model

he otts model viD PHHW is generliztion of the ssing model to more thn two sttes per pointF he otts model onsists of N spins tht re pled on lttieY t eh point i the spin tkes one of q possile vluesD distriuted uniformly out the irleD t ngles θ i = 2πN q , exemples re shown in pigure QFPF henD the intertion etween two neighoring spins i nd j is hrterized y the ngle Θ ij etween themF husD generliztion of @QFIA is given yX

U (ω) = -α i δ i (ω) -β i,j Θ ij (ω).
@QFPA pigure QFPX ystem of spins in the otts modelF his energy model is enountered in vrious (elds @eFgFD sttistil physisD multivrite sttistisD omintoril optimiztionD rti(il intelligeneAF e re here interested in its use in wrkovin priors s regulriztion termF 3.2 General characteristics of Markovian image models sn proility theory nd sttistisD wrkovin models refer to the (nite memory stohsti proessesF e wrkov hin is the simplest wrkovin model nd is widely used in oneEdimensionl signl proessingD queuing theoryD nd imge proessing s wellF Denition 3.2.1 (Markov chain) A Markov chain is a sequence of random variables X = (X i ) N i=1 that satisfy the Markov property, i.e., such that the distribution of the present state given the entire past can be restricted to the distribution conditioned only to the most recent past sample. Formally:

∀i > 1, p(X i | X i-1 , • • • , X 1 ) = p(X i | X i-1 ).
@QFQA ptil wrkov ndom pields @wpsA were formulted y fesgD IWUR nd hve eome populr imge proessing tools with qemn nd qemnD IWVRF hese models re sed on the de(nition of lol neighorhood systemF Denition 3.2.2 (Neighborhood system and clique) Let us denote as S anite set. The elements of S are named sites. Each site s ∈ S is associated with a set of neighbors V s ⊂ S such as:

• s / ∈ V s , ∀s ∈ S • s ∈ V t ⇔ t ∈ V s ,∀s, t ∈ S N = {V s , s ∈ S} is called a neighborhood system.
A subset c ⊆ S is a clique if every pair of distinct sites in c are neighbors. C denotes the set of cliques. In image processing applications, the set S of sites often coincides with the pixel lattice, and each site is consequently a pixel. In the case of region-based image analysis, the set S may include regions obtained by a segmentation algorithm. For example, in the former case, the set of neighbors of site s in a regular lattice S is commonly dened as

V s = {r, d(s, r) o},
where o is the order of the neighborhood system, and d() is the Euclidean distance. Figure 3.3 illustrates the neighborhood systems for o = 1 and 2 and their corresponding cliques.

∀s ∈ S, p(X s | X t , t = s) = p(X s | X t , t ∈ V s ). @QFRA
Local specications are given by the probabilities p(

X s = x s | X t = x t , t ∈ V s ).
he de(nition of wrkov (elds is very lolF xeverthelessD the suess of wrkov (elds is lrgely due to their pility to lso provide n expression of the glol distriution p(X)F e theoretil result out the equivlene etween wrkov rnE dom (elds nd qis distriutions is given y the Hammersley-Cliord theorem fesgD IWURF Theorem 3.2.1 (Hammersley-Cliord) Let us denote as X a random eld indexed by a set S with respect to the neighborhood system N X is a Markov random eld with respect to N ⇔ X is a Gibbs eld with respect to N Denition 3.2.4 (Gibbs elds) X is a Gibbs eld (GF) indexed by a set S with respect to the neighborhood system N if:

p(X = x) = 1 Z e -U (x) , @QFSA
where Z is a normalization constant (called partition function), and U n energy function that can be decomposed into a sum of potentials associated with the set of cliques C:

U (x) = c∈C V c (x). @QFTA
Equivalently, the joint distribution of x factorizes into a product of positive factor potentials:

p(x) = 1 Z c∈C g c (x), @QFUA
where g c = e -Vc .

Inference and Bayesian estimation

e now fous on the mthemtil formultion of wpEsed imge lssi(tionF he im of the lssi(tion is to estimte set of hidden lels X @eFgFD lnd over lss lelsA given set of oservtions Y @eFgFD stellite dtA tthed to the set of nodes @pixelsA S F ih lel oupies vlue in the set

Λ = {0, 1, • • • , M -1}F
he on(gurtion spe Ω = Λ |S| is the set of ll glol disrete lelingF x = {x s } s∈S nd y = {y s } s∈S denote reliztions of the rndom (elds of the lss lels nd oservtions of ll nodesF sn this ontextD we onsider the prolem of inferring the 4est4 on(gurtion x ∈ Ω given the oservtion yF his leds nturlly to n inferene prolem in whih the posterior distriution for the possile reliztion xD given the oservtions yD is omputed vi Bayes9s formul s X

p(X = x|Y = y) ∝ p(Y = y|X = x)p(X = x).
@QFVA sn @fFIAD p(X = x) is the prior distriution over on(gurtionsF sn other wordsD the distriution of the possile reliztions of X in the sene of ny oservtionsF por onvenieneD @fFIA is revited toX p(x|y) ∝ p(y|x)p(x). @QFWA imilr less umersome nottions will e used in the following whenever possileF eording to @QFWAD when oth the prior distriution p(x) nd the likelihood funtion p(y|x) re givenD the optiml solution tht n e estimted from these soures of knowledge is the fyes lelingF fyes9s theorem n only e pplied when ll these distriutions re suitleF roweverD it is not unommon for the resulting posterior to e vlid proility distriutionF sn this seD the posterior expeted loss @iFeFD fyes riskA is typilly well de(ned nd (niteX

R(x ) = x∈Ω C(x , x)p(x|y), @QFIHA
where C is the ost funtion penlizing the disrepny etween the estimted onE (gurtion nd the 4idel4 rndom on(gurtionF sndeedD he stndrd fyesin formultion of the inferene prolem onsists of minimizing the fyes risk whih is referred to s the generalized Bayes estimatorX

x opt = arg min x ∈Ω R(x ) = arg min x ∈Ω x∈Ω C(x , x)p(x|y). @QFIIA

Bayesian estimators emong the di'erent ost funtions employedD three hve een widely used in the litertureX

Maximum A Posteriori (MAP) estimate his lgorithm ims to extly estimte the mximum posteriori @weA on(guE rtionF he ost funtion of this lgorithm is de(ned y the followingX

C M AP (x , x) = 1 -δ(x , x), @QFIPA
where δ is the uroneker deltF his funtion hs the identil ost for ll on(guE rtions of x di'erent from xF prom iqutions @QFIHA nd @QFIPAD the fyes9s risk isX

R(x) = 1 -p(x|y). @QFIQA
winimizing @QFIQA is equivlent to mximizing the posterior proilityF hereforeD the miniml risk estimte isX

x M AP = arg max x∈Ω p(x|y) = arg max x∈Ω p(y|x)p(x), @QFIRA
sn dditionD if X is wrkovin proess nd ording to the wrkovEqis equivlene evoked in theorem @QFPFIAD the we estimte results in the minimiztion of n energy funtion sX

x M AP = arg min x∈Ω [U (y|x) + U (x)],
@QFISA where U (x) is the energy funtion ssoited with the wp XD nd U (y|x) = -lnp(y|x)F

Marginal a Posteriori Modes (MPM) estimate

he ost funtion ssoited with the ww is the followingX

C M P M (x , x) = s∈S [1 -δ(x s , x s )], @QFITA
whih is relted to the numer of sites F husD the ww riterion o'ers the possiE ility of penlizing errors ording to their numerD property tht mkes it espeE illy ppeling for imge segmenttion nd lssi(tionF he resulting fyesin estimtor is given y the followingX

∀s ∈ S, x M P M s = arg max xs∈Λ p(x s |y), @QFIUA
Mean Field (MF) estimate e third ost funtion tht my e onsidered is the wp estimtor using the following ost funtionX

C M F (x , x) = s∈S (x s -x s ) 2 , @QFIVA
he resulting fyesin estimtor is given y the followingX

∀s ∈ S, x M F s = x∈Ω x s p(x|y) = E[x s |Y = y], @QFIWA
whih is nothing else ut the onditionl expettion of X given Y = y whih exE plins the terminology 4men (eld4F xote thtD eh lel tkes vlues in the set Λ = {1, 2, . . . , M }F sn most usul pplitions to imge lssi(tion nd segmentE tionD these M possile lels re ment s symols tht indite M possile lssesF sn the se of the wp riterionD they re lso ment in their numeri sense s M integer numersF

Optimization approaches

qiven the fyesin justi(tion out the form of the ost funtion tht ws preE sented in the previous setionD we now turn into the mtter of how to otin the optiml solution in wrkovin frmeworkF por generl wpsD this prolem is xE hrd nd so (nding the optiml solution is fr from eing n esy tsk to hieveF es the full list of wp optimiztion tehniques is vstD we refer here only to seleted nd very smll suset of these methodsF e thorough review is reported in viD PHHWD uto nd eruiD PHIPF sn the followingD we will rie)y desrie the min optimiztion tehniques for wp modelsF Simulated annealing and Markov Chain Monte Carlo (MCMC) methods etullyD wgwg tehniques whih n e used for smpling ny type of distriuE tionF roweverD thnks to proedure nmed simulted nneling @ern yD IWVSD qo'e et lFD IWWRAD smpling n e lso used for optimiztion purposes with wp modelsF imulted nneling is inspired y n nlogy with metllurgyD in whih slow ooling @nnelingA is used to produe metl tht is tougher thn tht whih is produed from rpid oolingF hen wrkov hin simultion @typilly wetropolis smpler wetropolis et lFD IWSQA is used to smple qis distriuE tionD the nlogous proedure is to introdue temperture prmeter into the qis distriutionF he temperture is grdully redued from very high initil vlue to vlue lose to zeroF yn one hndD when T is lrge the smpler n move freely through the stte speD thus esping poor lol minimF yn the other hndD when the temperture is lose to zero the distriution is onentrted in the region ner the glol minimum nd so smpling eomes equivlent to optimiztionF he rtionle is thtD s is grdully deresedD one will e le to pproh the glol minimumF esymptotillyD simulted nneling gurntees to extrt the glol opE timum provided tht the temperture follows n pproprite deresing sheduleF hisD howeverD requires in(nite time in prtie nd so only lol minim n e usully estimtedF Iterated Conditional Modes (ICM) sterted gonditionl wodes @sgwA is nother deterministi optimiztion tehnique whih hs een proposed y fesg fesgD IWURD fesgD IWVTF sgw tries to mxiE mize the joint proility of n wp y mximizing lol onditionl proilities sequentillyF wore spei(llyD t eh step of the lgorithmD wp node is hosen @under rster snningA nd tht node is then ssigned the lel whih minimizes the energy of the wp under the ondition tht ll other nodes keep their lels (xedF his is repeted for ll nodes of the wp until onvergene iFeF until the energy nnot derese furtherF he min disdvntge of sgw is tht it is greedy nd very sensitive to initiliztionF hereforeD unless good initil estimte is given to sgwD it n esily get trpped to poor lol minimF righest gon(dene pirst @rgpA is deterministi lgorithm whih tries to improve sgw nd hs een proE posed y ghou nd frown ghou nd frownD IWWHF sts feture is tht it proesses the nodes of the wp in spei( orderF o this endD it introdues n unommitted lel nd then uses ertin strtegy for hoosing whih wp node 4to ommitt4 nextF ixperimentl results show tht rgp isD on the wholeD etter thn sgw with respet to the tsk of minimizing the energy of n wpF Graph-cuts qrph uts re onsidered e'etive pprohes for solving the energy minimiztion prolems in omputer visionF foykov nd tollyD PHHI proposed methods to solve weEwp using grphEut lgorithms nd they showed tht weEwp estimte is equivlent to minEut prolems on grph foykov nd uolmogorovD PHHIF his equivlene mkes grph uts extremely importntF hey showed tht using mxE )ow minEut grph lgorithms it is possile to solve some lss of energy funtions with weEwp frmeworkF

Other MRF optimization techniques fefore (nishing this setionD we mention some other tehniquesD whih hve lso een used in omputer vision for the optimiztion of wp methodsF enother lss of tehniquesD resemling the wy simulted nneling worksD re the soElled ontinution methods with grduted non onvexity @qxgA flke nd issermnD IWVU eing one suh exmpleF sn these methods the role of the temperture is plyed y nother prmeter γF sn this seD the intrtle nonEonvex energy funtion is pproximted y sequene of energy funtions prmeterized y this prmeter γF hen γ is lrge the energy funtion eomes stritly onvex nd so loting the glol optimum is esyF roweverD s γ deresesD the energy funtion eomes nonEonvex nd lol minim re strting to pperF he hope is tht if γ is grdully deresedD then y trking the sequene of lol minim we will e le to lote the glol optimum t the endF nfortuntelyD howeverD ontinution methods nnot provide ny optimlity gurntees out their solutionsD exept for ertin speil sesF enother tehniqueD whih hs lso een widely used is the relxtion leling method @sshikwD PHHQD pugers nd ferthodD IWVID osenfeld et lFD IWUTAF he ide of relxtion leling is to reple the disrete lels with ontinuous ones tht must lie on high dimensionl simplexF sn this wyD the prolem of optimizing disrete wp is onverted into onstrined ontinuous optimiztion prolemD whih n then e solved using stndrd grdient desent tehniquesF pinllyD we should note tht nother lss of optimiztion tehniques re the soElled geneti lgorithms @qolderg nd rollndD IWVVD rollndD IWUSAF hese re optimiztion methods whih re inspired y the priniple of nturl evolution in the iologil world nd try to simulte the evolutionry proessX in popultion of individulsD those who possess the highest goodnessEofE(t vlues re the ones who (nlly surviveF elthough geneti lgorithms re generl purpose optimiztion methods nd were found e'etive in mny pplitionsD their min disdvntge is tht they re mostly sed on heuristi proeduresF 3.4 Graphical representation of Markovian models he theory of wrkovin modelsD s prt of the theory of rndom proessesD shows strong onding with graphsF pormllyD grph G is pir of sets (S, E)D where S is the set of verties @nodesA nd E is the set of edgesD formed y pirs of vertiesF sn this frmeworkD it is nturl to desrie fmily of rndom vriles y set of sites tht onstitute the nodes of the grph tht represent the rndom proessF yne of the dvntges of using grphs in proilisti frmework is the possiility of immedite visul interprettion of the reltionships etween vriles tht expresses di'erent dependenies mong the nodesY tht is why we n tlk out dependeny grphsF he underlying proility distriutions of wrkovin models n e represented in grphil formD this is why these models re often lled proilisti grphil models fishopD PHHTF e proilisti grphil model is digrmmti representtion of proility distriutionF sn suh grph there is node for eh rndom vrileF he sene of n edge etween two vriles represents onditionl independene etween those vrilesF Denition 3.4.1 (Conditional independence) Conditional independence means that two random variables X and Y are independent given a third random variable Z (denoted as, X ⊥ Y | Z), i.e., if they are independent in their conditional probability distribution, formally :

p(X, Y | Z) = p(X | Z)p(Y | Z)
∀{s, t} ⊂ S (s, t) ∈ E ⇔ ∃c ∈ C : (s, t) ∈ c.
In other word, given a neighborhood system N = {V s , s ∈ S}: he lsses of usl utoregressive (elds vuettgen et lFD IWWRD unilterl wps hevijverD IWVVD mesh wps zlighi et lFD PHHWD ouse( nd uehtrnvzD PHIID nd mutully omptile wps on idimensionl ltties vferté et lFD PHHHD oisin et lFD PHIR hve thus een introduedF hese models rely on proilisti uslity onept ptured y the ftoriztion of p(x) in terms of usl trnsition kernels hittkerD PHHWF sn the following of this hpterD we shll rell the sis of uslity tht ould e pplied to wrkovin roessesF 3.5 Causality o introdue the onept of uslityD it is neessry to de(ne n order over the set of sites SF sn suh wyD we hrterize the pst of site sD s the set of nodes preeding sF por exmpleD is S is regulr twoEdimensionl lttieD the pst of site @pixelA s loted on the rth row nd c olumn n e de(ned s the set of ll the pixels loted from the (rst nd the (r -1) th rows nd of ll the pixels loted in the rth row from the (rst to the (c -1) th olumn hevijverD IWVV F wore exmples will e mentioned lter with regrd to hierrhil wpsF qiven the ordered set S = {1, • • • , s -1, s, s + 1, • • • , N }D the pst of the site s will e denoted s pa(s) = {i ∈ S : i < s}F husD the uslity is used to de(ne suset of sites ∂ s ⊆ pa(s) tht stis(es the following property @vuritzenD IWWTD hittkerD PHHWD vuritzen et lFD IWWHAX

t ∈ V s ⇔ (s, t) ∈ E.
∀s > 1, p(x s | x pa(s) ) = p(x s | x ∂s ).
@QFPHA sf @QFPHA holdsD one n rek the joint distriution p(x = x 1 , • • • , x N ) s produt of the onditionl proilities over the pst neighorhoodF pormllyX

p(x) = p(x 1 ) N s=2 p(x s | x pa(s) ) = p(x 1 ) N s=2 p(x s | x ∂s ) @QFPIA
he most remrkle implition of @QFPIA is the sene of normliztion onstntD whih helps to esily perform e0ient nd nonEitertive forwrd reursive smpling of this wrkov hinEtype distriutionF fy 4nonEitertive4D it is ment here tht solution x is otined diretlyD iFeFD without generting sequene {x k } ∞ k=0 of solutions tht onverge to xF sn the se of wrkovin proess indexed y n ordered set SD denoting s U (x) = c∈C V c (x) the energy funtion with the dependeny grph G = (S, N ) nd given the nie properties o'ered y uslityD the question tht rises is to whether we n determine if wrkov proess is usl on the sole sis of smll pst neighorhood ∂ s de(ned sX ∀s, ∂ s ⊆ V s ∩ pa(s)F fefore ontinuingD let us rerrnge the potentil produt c∈C g c suh sX

c∈C g c = 1≤i≤N g i D
where g i is the produt of g c 9s for ll c ontining i nd no further site j @iFeFD j > iAF the joint distriution @QFPIA eomesX

p(x) = p(x 1 ) N i=2 g i (x i , x ∂ i ), @QFPPA
where y de(nition of g i (x i , x ∂ i ), where

g i D ∂ i ⊆ V i ∩ pa(i)F roweverD
∂ i = V i ∩ pa(i). X is causal according to ∂ i , if ∀i, N i=2 g i (x i , x ∂ i )
is a constant with refer to x i .

Graphical characterization of the causality

qrphil onsidertions llow to point out n importnt lss of intertion models for whih the funtionl hrteriztion of the uslity systemtilly holdsF sn ftD the grphil viewpoint llows in some ses to identify t (rst glne @without need of ny omputtionl or proilisti rgumentA intertion strutures tht support usl modelsF Proposition 3.5.2 (Graphical characterization) We denote by X a Markovian process admitting G as an independence graph with the joint distribution p(X

= x) = p(x 1 ) N i=2 g i (x i , x ∂ i ), where ∂ i = V i ∩ pa(i). X is causal according to ∂ i , if ∀i, ∂ i = V i ∩ pa(i) is a complete graph 2 .
eginD the ene(t of dependeny grph is highlighted euse it llows for visul immedite identi(tion of ertin usl modelsF sn ftD it hs een shown in hittkerD PHHW tht grphs stisfying the grphil hrteriztion in ropoE

f (i, j) = p(X t = i|X t-1 = j); e(k, l) = p(X t = k|Y t = l) @QFPQA
por the ske of rgumentD ll rndom vriles in @QFPQA re ssumed disrete ut extensions to the ontinuous oservtion vriles y re rther strightforwrdF feE use of its usl ntureD the grph generted y wrkov hin llows otining the posterior distriution of the hidden vriles e0iently using two stge mesE sge pssing lgorithmF sn the prtiulr ontext of the hidden wrkov modelD this is known s the forwrdEkwrd @pEfA lgorithm whih is n exmple of dynamic programing 3 iner et lFD IWVWF here re severl vrints of the si pEf tehnique tordnD PHHQF sn the followingD we shll fous on the most widely used lgorithmsF

Viterbi algorithm he generl form of the pEf lgorithm n e used to do inferene on individul sttesD whih represent hidden vrilesD of the wrkov hin given sequene of oservtionsF roweverD in mny pplitionsD the hidden vriles hve some menE ingful interprettionsD so it is often of interest to (nd the most prole sequene of hidden sttes for given oservtion setF his is wht the iteri lgorithm does @iteriD IWTUD hvidEporneyD IWUQAF pormllyD the gol of this lgorithm is to ompute reursively the est sequene x of the hidden vriles tht mximize the posterior distriutionX . . , Y t = y t ) nd where the lst stte is equl to x t @X t = x t AF henD one ould introdue th following quntityX

x = arg max x p(X = x|Y = y) @QFPRA = arg max
δ t (x t ) = max x 1 ,...,x t-1 p(Y 1 = y 1 , . . . , Y t = y t , X 1 = x 1 , . . . , X t-1 = x t-1 , X t = x t ),
@QFPSA por onvenieneD @QFPSA is revited toX

δ t (x t ) = max x 1 ,...,x t-1 p(y 1 , . . . , y t-1 , x 1 , . . . , x t-1 , x t ), @QFPTA
whih represents the proility of the est pth from those previously onsideredF xextD under onditionl independene ssumptions derived from the grph in pigE ure QFV nd using roposition QFSFQ one ould ompute @QFPSA reursively s followsX = max

δ t (x t ) = max x 1 ,...,
x t-1 [e(y t , x t )f (x t , x t-1 )δ t-1 (x t-1 )],
whih gives the following reursionX δ t (x t ) = e(y t , x t ) max

x t-1 [f (x t , x t-1 )δ t-1 (x t-1 )], @QFPVA
where δ 1 (x 1 ) = p(y 1 , x 1 ) = p(x 1 )e(y 1 , x 1 ) @QFPWA whih gives

x t = arg(δ t (x t )) @QFQHA x t = arg max x t-1 [f (x t , x t-1 )δ t-1 (x t-1 )] Algorithm 3.5.1 (Viterbi Algorithm) Initialization δ 1 (x 1 ) = p(x 1 )e(y 1 , x 1 ).
⇒ Forward pass ∀t ∈ {2, . . . , n}:

δ t (x t ) = e(y t , x t ) max x t-1 [f (x t , x t-1 )δ t-1 (x t-1 )] x t = arg max x t-1 [f (x t , x t-1 )δ t-1 (x t-1 )]
⇐ Backward pass Construction of the optimal path x as a sequence of the optimal states computed during the forward step.

Baum-Welch algorithm fum et lFD IWUH tried to solve the prolem of estimting the optiml sequene sed on likelihood riterion X

L t = p(x t , y) @QFQIA x t =
F 1 = p(x 1 , y 1 ) = e(x 1 , y 1 )p(x 1 )
Recursion ∀t ∈ {2, . . . , n}:

F t = e(y t , x t ) x t-1 f (x t , x t-1 )F t-1 ⇐ Backward pass Initialization B n = 1 x n = arg max xn F n
Recursion ∀t ∈ {n -1, . . . , 2}:

B t = x t+1 f (x t+1 , x t )e(y t+1 , x t+1 )B t+1 x t = arg max xn B t × F n
sn the following hpterD we will fous on usl models on wrkov (elds vi the prtiulr se of tree strutures s they re tringulted y de(nition nd do not inlude ylesF

Chapter 4

Hierarchical Markov Models on quad-trees pigure RFPX @A e typil exmple of treeY @A tree struture tht models the intertion etween imges in the pyrmid strutureF in monoEdimensionl nd iEdimensionl signlsD respetivelyF es disussed in the setion QFSFP of the previous hpterD the seletion of these strutures is justi(ed y their uslity properties over sle nd y the possiility of employing fst optimiztion methodF purthermoreD the inferene tehnique using tree strutures is muh simpler thn the tehniques for other tringulted grphs euse of the yli @iFeFD loopEfreeA propertyF e useful interprettion mkes use of the reltionship etween direted grphil models nd undireted onesF sn direted grphil modelD the quntities tht must e spei(ed inlude the onditionl distriution t eh node given the vlues of ll of its prents @where s -is prent of s if there is direted edge from s -to sAF st is strightforwrd to onvert direted grphil model into n undireted oneD ut the onstrution of direted grphil model equivlent to n undireted one is generlly very omplex ndD in ftD requires de(ning new node nd edge sets where the nodes onsist of entire liques of nodes of the originl undireted grphF por treeD howeverD the onstrution of direted grphil model from n undireted one is trivil nd in ft does not hnge the nodes of the grph nor the grphil struture @exept tht edges eome direted rther thn undiretedAF pei(llyD onsider n undireted grphil model over tree nd hoose ny node to designte s the 4root4 nodeF gonsider then 4hnging4 the tree from this node iFeFD redrw the grph with the root node t the top levelD with its neighors t the next levelD etF por exmpleD in pigure RFP @AD we hve leled one nodeD s -D in pigure RFP @A s the root node of the onsidered treeF vet us denote generi node on the spei( se of qudEtree s s nd the (nite set of ll nodes s S(s ∈ S)F ih node is pixel in one of the levels of the treeF he set of nodes is then hierrhilly prtitionedD @iFeFD S = S 0 ∪ S 1 ∪ . . . ∪ S R A where S n indites the suset of nodes ssoited with the n th level of the tree (n = 0, 1, . . . , R)D n = R denotes the root of the tree @orsest resolutionA nd n = 0 indites its leves @(nest resolutionAF sn the onsidered strutureD prentEhild reltionship n e de(nedX n upwrd shift opertor δ suh tht s -= δ(s) is the prent of node sF he opertor δ is not oneEtoEoneD ut fourEtoEone euse eh prent hs four o'spring @euse of the qudEtree strutureAF e de(ne the forwrd shift opertor β suh tht s + = β(s) is the set of ll the desendnts of sD the interhnge opertor α is de(ned s etween the nodes in the identil sleD nd d(s) is the set inluding s nd ll its desendnts in the tree s illustrted in pigure RFP @AF his frmework llows dt t di'erent resolutions nd di'erent spetrl nds to e fusedF 4.2 Estimation and inference algorithm on quad-tree 4.2.1 Problem statement he desried hierrhil struture llowsD in nturl wyD the use of n expliit sttistil model through hierrhil wrkov rndom (eld formultion using set of rndom (eldsD whih re ssoited with the di'erent slesD nd exploiting the opertors de(ned ove on the qudEtree strutureF hereforeD let us denote the lss lel of site s s disrete rndom vrile x s nd its vlue s ω s @s ∈ SAF sf there re M lsses in the onsidered seneD then eh lel oupies vlue in the set Λ = {0, 1, . . . , M -1} @iFeFD x s , ω s ∈ ΛAF he lss lels of ll pixels n e olleted in set X = {x s } s∈S of rndom (elds X n = {x s } s∈S n ssoited with eh sle nD where S n is the relted set of lttie pointsF he orresponding on(gurtion t sle n nd n e represented s ω n = (ω s ) s∈S n F he on(gurtion spe Ω = Λ |S| is the set of ll glol disrete lelings @iFeFD X ∈ ΩAF yne ould then ssume the following to (t wp model to the forementioned hierrhil strutureX (i) he fundmentl ssumption of the model is tht the sequene of rndom (elds from orse to (ne sles forms wrkov hin over sle X

p(X n = ω n |X q = ω q , q > n) = p(X n = ω n |X n+1 = ω n+1 ).
@RFIA por onveniene @RFIA is revited toX

p(x n |x q , q > n) = p(x n |x n+1 ). @RFPA
(ii) he trnsition proilities of this wrkov hin ftorize so tht the ompoE nents @pixels or node sA of X n re mutully independent given the omponents of X n+1 X

p(X n = ω n |X n+1 = ω n+1 ) = s∈S n p(x s = ω s |x s -= ω s -). @RFQA
egin for onveniene purpose @SFPA is revited toX

p(x n |x n+1 ) = s∈S n p(x s |x s -). @RFRA
sing the qudEtree struture llows ene(ting from the good properties disE ussed in the etion RFI @iFeFD uslityA nd pplying nonEitertive lgorithmsD thus resulting in derese in omputtionl time ompred to itertive optimizE tion proedures over grphsF es shown in ghpter QD severl fyesin pprohes re o'ered to solve this inferene prolem y minimizing the fyes risk @see QFIHAF emong the di'erent lssi(tion lgorithms employed on qudEtree struture in the litertureD two hve een widely used nd mth the we nd ww riteriD respetivelyF he (rst lgorithm ims to extly estimte the we on(gurtionD the resulting estimtor is given y QFISF he seond lgorithm produes the on(gurtion tht mximizes t eh site s the posteriori mrginlD the resulting estimtor is given y QFIUF o etter highlight the di'erene etween these two riteri nd their impliE tions in the pplition to qudEtreesD let us rell the formultion of the ost funE tions ssoited with the we nd ww @see equtions QFIP nd QFIT in ghpter QAF he ost funtion of we is de(ned y the followingX

C M AP (x , x) = 1 -δ(x , x) = 1 - s∈S δ(x s , x s ).
@RFSA his funtion implies identil ost for ll on(gurtions of x @di'erent from xAF the we estimtor of the lel (eld X is given y the followingX

x M AP = arg max x∈Ω p(x|y), @RFTA
his omintoril optimiztion prolem n e resolved y using ulmnE like (lter illskyD PHHPD owing to the forml similrity etween wp models nd the iteri lgorithm hvidEporneyD IWUQF he extension of the iteri lgorithmD whih omputes the ext we estimte of X given Y = y on the qudEtree hs een introdued y hwidD IWWP @see etion QFQFIA in the ontext of proilisti expert systemsD nd vferté et lFD PHHH in the ontext of imge lssi(tion y proposing nonEitertive lgorithm on the qudEtreeF roweverD in the pplition to qudEtreesD these lgorithms exhiit two min shortomingsF pirstD omputtionllyD they re known to e 'eted y under)ow prolems euse of the smll proilities involvedF eondD ording to @RFSAD the we ost funtion penlizes the disrepnies etween on(gurtions regrdless of their orresponding slesD n undesirle property from the viewpoint of segmenttionD lelingD nd lssi(tionF pei(llyD n error t orser sle will e pid the sme ost s n error t (ner sle wheres it is desirle to hve higher ost for errors t orser levels euse they my generlly led to the mislssi(tion of groups of pixels t level 0 @eFgFD one pixel t the root orresponds to 4 R pixels t the (nest sleAF yn the ontrryD the ww riterion is sed on riterion funtion tht ims t segmenttion ury nd llows errors on distint sles to e penlized di'erently vferté et lFD PHHHD uto nd eruiD PHIPF he ost funtion isX

C M P M (x , x) = s∈S [1 -δ(x s , x s )],
@RFUA whih is relted to the numer of sites s ∈ S suh tht x s = x s F he ww riterion penlizes errors ording to their numerD t the sle t whih they ourF he fyesin estimtor resulting from @RFUA is given y the followingX ∀s ∈ S, x

⇓ Top-down pass

This downward recursion reassembles the complete posterior marginals using (4.9) and then deduces the MPM solution via (4.8).

4.3 Blocky artifacts and further extensions of the hierarchical model

Blocky artifacts

he tree struture indues nonEsttionrity in speX the distriution t the leves is not invrint with respet to the δ opertorD sine the orreltion etween two vriles depends on the 4distne4 to their ommon nestor in the treeF his my result in 4loky4 rtifts in the (nl lssi(tion mpF his phenomenon n e explined y the ft tht two neighoring sites t given sle my not hve the sme prentF sn this seD oundry is more likely to pper thn when they re linked y prent nodeF his rtift hs een reported in severl ppers foumnD IWWID ghou et lFD IWWQD vuettgen et lFD IWWRD pieguth et lFD IWWVD vferté et lFD PHHHF pigure RFR shows the loky rtifts where the method in vferté et lFD PHHH ws pplied using the ww riterion t vrying levels of the qudEtree (R = 1, 2, 3) using léides imgesF everl tehniques hve een proposed to irumvent suh undesired e'etsF por exmpleD posteriori smoothing ws developed in vuettgen et lFD IWWR usE ing multiEsle regulriztion @wA lgorithm ghouD IWWIF enother wy to lE levite these e'ets ws to de(ne tree struture with overlpping dt leves srving et lFD IWWUF purther methods onsists in using hierrhil grph struE tures tht re more omplex thn usul trees foumnD IWWID uto et lFD IWWTD gomer nd helpD IWWWF nfortuntelyD in these sesD the prtil dvntges of the tree struture re then prtly or ompletely lostF pinllyD other sienE tists voided the lok rtifts y using semiEitertive lgorithms in whih the prior distriution ws omputed itertively on the qudEtree @prior smoothingA ghrdinD PHHHD oisinD PHIPF ome of these lgorithms re desried in the next setionF his results in n interleved model whose mnipultion is itertiveF he inferE ene is onduted using prllel relxtion sheme fesgD IWVT whih is sed on modi(ed nneling proedure @multiEtemperture nneling uto et lFD IWWTA where temperture is kept high t the orse levels of the strutureD thusD enling the lgorithm to e less sensitive to lol minimF his pproh provides urte lssi(tion results ut is omputtionlly demndingF

Chardin et al.'s method

ghrdinD PHHHD ghrdin nd érezD IWWW investigted n extension of the hierrhiE l stohsti model desried in vferté et lFD PHHH y tking ene(t from oth the sptil nd the hierrhil prior modelD omine usl hierrhil prior on trees with nonEusl sptil prior t the orsest level of the hierrhyX it is wrkov rndom (eld tthed to the nodes of trunted tree s shown in pigure RFTF he proposed hyrid struture leds to semiEitertive lgorithms whih mix ext nonEitertive proedures on suEtrees nd itertive proedures on the grid t the top of the strutureF (a) Quad-tree (b) Hybrid structure pigure RFTX wo hierrhil struturesX @A qudEtree with three levelsY @A trunE ted tree with two levels proposed y ghrdinD PHHHD ghrdin nd érezD IWWWF

Voisin et al.'s method

oisinD PHIPD oisin et lFD PHIR enhned the lssi(tion tehnique in vferté et lFD PHHH y etter estimting the prior t the orsest sle through seEspei( prior updte lgorithmF husD ww estimtion on REsle tree is performed (rst y hoosing n uniform priorD nd then y using the preliminry results s new priorF xextD smller tree of sle R -1 is onsidered to whih the ww lgorithm is pplied to estimte new priorF his step is repeted itertively until sle H is rehedF he sheme of this lgorithm is given in pigure RFUF his method improves roustness with respet to loky rtifts s ompred to the originl method in vferté et lFD PHHHD thus inresing the ury when ompred to the si ww tree s experimentlly demonstrted in oisin et lFD PHIRF pigure RFUX ww estimtion on the qudEtree s proposed in oisinD PHIPD oisin et lFD PHIR sn this representtionD a PF he min dvntge of lgorithms whih use qudEtree strutures is their E pilities to uilt nonEitertive proedures when the uslity property holdsF sn prtiulrD reursive proedure on qudEtree is fesile for the mrginl posteE rior modes @wwA riterionF hen the im is to mximize the posterior mrginl t eh site sF feuse the tree is yliD the lels re estimted reursively through forwrdEkwrd lgorithm similr to the lssil fum nd elh tehnique for wrkov hin desried in lgorithm QFSFQF sn prtieD the use of one qudEtree struture with the ww riterion often yields 4loky4 e'ets s shown in pigE ure RFRF his phenomenon n e explined y the ft tht two neighoring sites t given sle my not hve the sme prentF his undesirle e'et n e reE dued using some modi(tion s in the lgorithms desried in the previous setionF sn the next hptersD severl other hierrhil wp models sed on qudEtree will e introdued to support multiEresolutionD multiEtemporlD multiEsensor dtD nd to inorporte sptil ontextul info either t the root or in ll levels of the treesF Chapter 5

Multi-temporal hierarchical Markov model on quad-tree 5.1 Multi-temporal structure sn the previous hpter @setion RFIA the si qudEtree model ws desriedF e generi node ws denoted s s nd the (nite set of ll nodes ws denoted s S(s ∈ S)F sn suh struture @see pigure RFPAD prentEhild reltionship ws de(nedX δ: is the upwrd shift opertor suh tht s -= δ(s) is the prent of node sF β: is the forwrd shift opertor suh tht s + = β(s) is the set of the desendnts of sF α: is the interhnge opertor etween the nodes t the sme sleF d(s): is the set inluding s nd ll its desendnts in the treeF his frmework llows dt t di'erent resolutions nd di'erent spetrl nds to e fusedF e novel element of the (rst proposed pproh whih is methodologilly desried nd experimentlly vlidted in this hpterD is the multiEtemporl spetF hereforeD multiple pyrmids nd qudEtrees re employed in sdeD eh pyrmid eing ssoited with the set of imges ville t di'erent dte to hrterize the temporl orreltions ssoited with distint imges in the input time seriesD while still supporting the lssi(tion of dt quired t multiple sptil resolutionsF por this purposeD this extended multiEtemporl qudEtree struture is endowed with new opertors to link etween the nodes ross di'erent dtesD nd is omined with multiEsle wvelet feture extrtionF e de(ne n upwrd shift opertor suh tht s = = (s) is the prent of node s in the previous dte of the time seriesF purthermoreD we de(ne n interhnge opertor σ etween the nodes t the identil sle nd identil position ut from onseutive dtes to hrterize the temporl orreltion etween imges given t di'erent dtes @see pigure SFIAF his multiEtemporl hierrhil struE ture is imed t supporting the joint lssi(tion of oth multiEtemporl nd multiEresolution input imgesF his implies thtD if only one imge is ville on ertin quisition timeD then it will e inluded t the (nest resolution lyer @level HA of the orresponding qudEtreeF reneD the intrinsi resolution of the imge will e the (nest resolution of the qudEtree of tht dteF ell other levels of the tree will e (lled in using wvelet trnsforms wlltD PHHV of the input imgeF sf imges orresponding to multiple resolutions re ville on ertin timeD senrio tht oursD for exmpleD when there re oth higher resolution pnhroE mti nd orser resolution multiEspetrl dtD then eh singleEresolution input imge is inluded in seprte level of the qudEtreeF sn this seD n ssumption impliit in the qudEtree topology is tht the sptil resolutions of the input imE ges re relted y powerEofEP reltionshipF his ondition is stis(ed with minor pproximtions y most urrent multiEresolution speEorne optil sensors @see ghpter IAD so it is urrently only mild restrition from n opertionl viewpointF pigure SFIX wultiEtemporl hierrhil strutureF efter the input imges re inluded in the lyers orresponding to their sptil resE olutionsD level H of the qudEtree orresponds to the (nestEresolution input imgeD while some intermedite levels generlly remin empty nd re (lled in using wvelet trnsforms of the imges from the lower @(ner resolutionA lyersF por exmpleD if n suyxy quisition omposed of pnhromti omponent t IEm resolution nd multiEspetrl omponent t REm resolution is used on ertin dteD then the (nest resolution of the qudEtree is I mD the pnhromti nd multiEspetrl imges re inluded t levels H nd PD respetivelyD nd level I is omputed s wvelet trnsform of the pnhromti dt t level HF he proposed hierrhil struture llowsD in nturl wyD the use of n expliit sttistil model through hierrhil wrkov rndom (eld formultion using seE ries of rndom (elds t vrying sles nd timesD using the opertors de(ned oveD on the onseutive qudEtreesF vet us denote the lss lel of site s s disrete rndom vrile x s nd its vlue s ω s (s ∈ S = ∪S n t , where t nd n refer to the orresponding time nd sle respeE tively )F sf there re M lsses in the onsidered seneD then eh lel tkes vlue in the set Λ = {0, 1, . . . , M -1}F he lss lels of ll pixels n e olleted in set X = {x s } s∈S of rndom (elds X n t = {x s } s∈S n t ssoited with eh sle n nd dte tD where S n t is the relted set of lttie pointsF he orresponding on(gurtion t sle n nd dte t n e represented s ω n t = {ω s } s∈S n t F he on(gurtion spe Ω = Λ |S| is the set of ll glol disrete lelings @iFeFD X ∈ ΩAF e then ssume the following to (t n wp model to the forementioned hierrE hil strutureX (i) he fundmentl ssumption of the model is tht the sequene of rndom (elds from orse to (ne sles form wrkov hin over sle nd timeX

p(X n t = ω n t |X q p = ω q , p < t, q > n) = p(X n t = ω n t |X n+1 t = ω n+1 t , X n+1 t-1 = ω n+1 t-1 ). @SFIA
(ii) he trnsition proilities of this wrkov hin ftorize so tht the ompoE nents @pixels or nodes sA of X n t re mutully independent given the omponents of

X n+1 t nd X n+1 t-1 X p(X n t = ω n t |X n+1 t = ω n+1 t , X n+1 t-1 = ω n+1 t-1 ) = s∈S n p(x s = ω s |x s -= ω s -, x s = = ω s = ).
@SFPA sn the next setionD n extended version of the ww estimtor will e disussed using the proposed multiEtemporl hierrhil strutureF 5.2 Multitemporal MPM inference sn the proposed pprohD the posterior mrginl p(x s |y) of the lel of eh sptioE temporl node s is expressed s funtion of the posterior mrginl p(x s |y) of the prent node s -in the orresponding qudEtree nd the posterior mrginl p(x s |y) of the prent node s = in the qudEtree ssoited with the previous dte to hrterize the temporl orreltions ssoitedD t di'erent slesD with distint imges in the input time seriesF he posterior mrginl of eh sptioEtemporl site s n e written s followsX

p(x s |y) = x s -,x s = p(x s , x s -, x s = |y d(s) ) xs p(x s , x s -, x s = |y d(s) ) p(x s -|y)p(x s = |y) @SFQA
where old fonts denote the mrginl posteriors of interest to the wwF Proof of equation (5.3)X

p(x s |y) = x s -,x s = p(x s |x s -, x s = , y)p(x s -, x s = |y) = x s -,x s = p(x s |x s -, x s = , y d(s) )p(x s -, x s = |y) = x s -,x s = p(x s , x s -, x s = |y d(s) ) xs p(x s , x s -, x s = |y d(s) ) p(x s -, x s = |y) = x s -,x s = p(x s , x s -, x s = |y d(s) ) xs p(x s , x s -, x s = |y d(s) ) p(x s -|y)p(x s = |y),
he equlities ross rows I nd PD nd ross rows Q nd RD derive from two onditionl independene ssumptionsX A1. he lel x s D given the lels of the prents x s -nd x s = t the sme nd the previous dtesD depends only on the oservtions y d(s) of site s nd of its desendnts nd not on the oservtions of the other sitesD iFeFD

p(x s |x s -, x s = , y) = p(x s |x s -, x s = , y d(s) )
A2. qiven the oservtions yD the lel of the prent s -of site s t the sme dte is independent on the lel of the prent s = t the previous dteD iFeFD

p(x s -, x s = |y) = p(x s -|y)p(x s = |y)
hese ssumptions re nlogous to the onditionl independene ssumptions tht re ommonly epted when deling with @hierrhil or singleEsleA wpEsed imge nlysisD nd hve een mentioned in the previous hptersF hey re used within the proposed method for nlytil onvenieneF he formultion in @SFQA llows lulting reursively the posterior mrginl p(x s |y) t eh sptioEtemporl node s while the proilities p(x s , x s -, x s = |y d(s) ) re produedF husD this simpli(es to the determintion of the other proilities sed on the following equtionX

p(x s , x s -, x s = |y d(s) ) = p(x s |x s -, x s = ). p(x s -|x s = )p(x s = ) p(x s ) p(x s |y d(s) ) @SFRA
sn @SFRAD the (rst ftor p(x s |x s -, x s = ) orresponds to the hildEprent trnsition proilityY p(x s ) is the prior proilityY p(x s -|x s = ) is the temporl trnsition proility t the sme sleY nd p(x s |y d(s) ) is the prtil posterior mrginl proilityF Proof of equation (5.4)X

p(x s , x s -, x s = |y d(s) ) = p(x s -, x s = |x s , y d(s) ) p(x s |y d(s) ) = p(x s -, x s = |x s ) p(x s |y d(s) ) = p(x s |x s -, x s = ) p(x s -, x s = ) p(x s ) p(x s |y d(s) ) = p(x s |x s -, x s = ) p(x s -|x s = )p(x s = ) p(x s ) p(x s |y d(s) )
where the equlity ross rows I nd P derives from the following ssumptionX A3. he distriution of the lels x s -nd x s = of the prents of site s re indeE pendent on the oservtions y d(s) of the desendnts of sD when onditioned to the lel x s of sD iFeFD

p(x s -, x s = |x s , y d(s) ) = p(x s -, x s = |x s )
o ompute these proilitiesD we ene(t from the hierrhil struture de(ned ove nd use three reursive psses on the qudEtreeD inluding one bottom-up nd two top-down pssesF por the ske of revityD only the steps ssoited with pir of imges quired t two di'erent times @t = 0 nd t = 1A re explined in the following @see pigure SFPAF he reursive extension to more thn two quisition times is strightforwrdF 5.2.1 Time t = 0: single-time MPM.

eording to the sde pprohD (rstD lssi(tion is performed t time taH using singleEdte ww s shown in hpter RD in whih the leling is otined reursively over sles through topEdown stge nd ottomEup stgeF hetils of this singleEdte formultion n e found in vferté et lFD PHHH nd hs een relled in etion RFPF e only rell tht the proess is initilized y prede(ning the pixelwise prior proility distriution t the root of the orresponding qudE treeD iFeFD p(x s )D s ∈ S R 0 F his initiliztion is required to egin topEdown reursion nd ompute the priors t ll levels of the qudEtree t time 0F e simple initilE iztion strtegy is to use uniform prior distriution on ΛF rereD to inorporte sptil ontextul informtion nd mitigte possile loky rtiftsD seEspei( initiliztion strtegy is pplied tht mkes use of sptil wp modelX neighE orhood system is de(ned on the lttie S R 0 in the root t time HD nd for eh pixel s ∈ S R 0 D the unonditionl prior p(x s ) is repled y the lol onditionl prior p(x s |x s , s ∼ s, s ∈ S R 0 )D where s ∼ s denotes tht the sites s nd s re neighorsF his hoie generlly provides ised priorEproility estimte ut fvors sptil dptivityD desired property when working with high resolution imges in whih sptil detils re ommonF he wellEknown otts wp modelD whih fvors the sme leling in homogeneous imge regionsD is used viD PHHWD uto nd eruiD PHIPD iFeFX

p(x s |x s , s ∼ s, s ∈ S R 0 ) ∝ exp(β s∼s δ(x s , x s )) @SFSA
where β is positive sptil smoothness prmeterF everl methods hve een proposed to optimize the vlue of this prmeter inluding the mximiztion of the pseudoElikelihood funtion over the trining set qemn nd qemnD IWVRF sn ghrdinD PHHH lso omined hierrhil struture nd the otts modelD leding to semiEitertive tehnique in whih the otts omponent is used to ompute unique prior distriution for eh sleF yn the ontrryD hereD we use the otts model to de(ne lol hrteristi for eh node of the root level to mximize sptil dptivityF es result of singleEtime proessing t time t = 0D the posterior mrginl p(x s |y) is known for eh pixel of the orresponding qudEtreeY p(x s |y d(s) )D in whih y d(s) denotes the olletion of the oservtions of ll qudEtree sites tht re desendnts of site sD is lso derived s yEprodut (s ∈ S n 0 , n = 0, 1, . . . , R)F hetils n e found in vferté et lFD PHHH nd in etion RFPF 5.2.2 Time t = 1: rst top-down pass.

sn the proposed methodD the reursive topEdownGottomEup formultion used for the singleEtime se in ghpter R is extended to the multiEtemporl lssi(tion t time t = 1F sn this se s wellD (rstD the prior distriution on the root lttieD iFeFD p(x s ), s ∈ S R 1 D hs to e de(ned to initilize topEdown pssF pollowing the sde pprohD t time t = 1D we tke ene(t of the inferene onduted t time t = 0X for eh pixel s ∈ S R 1 on the root lttie t t = 1D the unonditionl prior p(x s ) is initilized s the posterior mrginl p(x σ(s) |y d[σ(s)] )D whih orresponds to the sme pixel σ(s) ∈ S R 0 in the root lttie S R 0 t t = 0 @lue rrow leled with the numer one in pigure SFPA nd hs een omputed s yEprodut of the singleEdte ww pplition t time t = 0F efter initilizing the prior in the rootD topEdown pss @lue rrow leled with the numer two in pigure SFPA is performed for eh (ner level n < R t time t = 1F he priorEproility distriution is derived s funtion of the priorEproility distriution t the prent level nd of the trnsition proilities from the prent to the urrent level (s ∈ S n 1 , n = 0, 1, . . . , R -1)X p(x s ) =

x s - p(x s |x s -) p(x s -). @SFTA his derivtion fvors n identil prentEhild leling nd models the sttistil intertions etween onseutive levels of the qudEtreeF e model the trnsition proility in the form introdued y foumnD IWWID iFeFD (s ∈ S n 1 , n = 0, 1, . . . , R-1)X

p(x s |x s -) =    θ x s = x s - 1 -θ M -1 x s = x s - @SFUA
where θ is prmeter rnging in [ 1 M , 1]F es result of the (rst topEdown pssD the prior distriution p(x s ) is derived for eh pixel s (s ∈ S n 1 , n = 0, 1, . . . , R) of eh level of the qudEtree t time t = 1F 5.2.3 Time t = 1: bottom-up pass.

e ottomEup pss reursion is then performed to estimte the joint proilities p(x s , x s -, x s = |y d(s) ) strting from the leves of the qudEtree t time t = 1 nd proeeding until the root is rehed sed on the ftoriztion in eqution @SFRAF sn ddition to priorsD whih hve een omputed in the previous topEdown pssD three sets of proilities re required to ompute this ftoriztionX (i) the set of temporl trnsition proilities t the sme sle p(x s -|x s = )Y (ii) the hildEprent trnsition proility p(x s |x s -, x s = )Y (iii) the prtil posterior mrginls p(x s |y d(s) )F hetils of the lultion of @iA nd @iiA re shown in etion SFQFPF gonerning @iiiAD vferté et lFD PHHH proved the following (s ∈ S n 1 , n = 1, 2, . . . , R)X p(x s |y d(s) ) ∝ p(y s |x s )p(x s )

u∈s + xu p(x u |y d(u) ) p(x u ) p(x u |x s ), @SFVA
husD the ottomEup pss is reursion tht estimtes p(x s |y d(s) )F st strts from the leves of the qudEtree in whih the prtil posterior mrginls re omputed vi @green rrow leled with the numer 1 in pigure SFPAX p(x s |y s ) ∝ p(y s |x s )p(x s ), @SFWA nd then proeeds until the root is rehed using eqution @SFVA @green rrow lE eled with the numer two in pigure SFPAF iqution @SFVA involves the pixelwise lssEonditionl hps p(y s |x s ) of the imge dt t eh node of eh qudEtree @see etion SFQFIAF es result of the ottomEup pssD we now hve ll needed proilities to ompute p(x s , x s -, x s = |y d(s) ) t eh level of the qudEtreeF 5.2.4 Time t = 1: second top-down pass.

eording to eqution @SFQAD (rstD the posterior mrginl is initilized t the root of time t = 1 @red rrow leled with the numer one in pigure SFPAF por this purposeD we initilize p(x s |y) s p(x s |y d(s) ) for s ∈ S R 1 D s in the usul singleEdte formultions of wwF henD the posterior p(x s |y) t eh pixel s for ll other tree levels t time t = 1 (s ∈ S n 1 , n = 0, 1, . . . , R -1) n e esily omputed reursively in topEdown pss @red rrow leled with the numer two in pigure SFPA using the formultion in eqution @SFQAF 5.2.5 Both times: combination with MMD. et eh time t ∈ {0, 1}D the forementioned steps led to the omputtion of the posterior mrginl p(x s |y) on eh pixel (s ∈ S n t , n = 0, 1, . . . , R)F sn prinipleD the lss lel x s tht mximizes p(x s |y) over the (nite set Λ of lsses ould e seleted nd ssigned to sF his is fesile proedure ut is often voided in the literture of hierrhil wps euse of its omputtionl urden @liner with respet to the numer of lsses nd the numer of sites for ll sles nd timesA nd of possile loky rtiftsF es n lternte pprohD hereD seEspei( formultion of the modi(ed wetropolis dynmis @wwhA utoD IWWS is pplied seprtely for eh sle nd timeF pei(llyD in the se of the root lyer of the qudEtree orresponding to eh time tD wwh is used to minimize the following energy with respet to the lel on(gurtion

X R t = {x s } s∈S R t X U (X R t |y) = - s∈S R t log p(x s |y) -β s∼s ∈S R t δ(x s , x s ) @SFIHA
where the (rst term is expressed in terms of the pixelEwise posteriors omputed y ww nd the seond ontriution is due to the otts model on the root of the treeF sn the se of eh other lyer n = 0, 1, . . . , R -1D no otts model is used nd wwh is pplied to minimizeX

U (X R t |y) = - s∈S R t log p(x s |y) @SFIIA
his mens thtD in this seD wwh is equivlent to itertively seleting rnE dom suset of pixels for whih rndom replements in lss memership re tE temptedF sn ll sesD the itertive proedure of wwh is repeted until the di'erene in energy on onseutive itertions goes elow prede(ned threshold @whih ws set to 10 -4 in the experimentsAF sn the se of the root lyerD the solutions otined using wwh nd mximizing p(x s |y) diretly intrinsilly di'er euse the former tkes into ount sptil ontext through the otts model while the ltter does notF sn the se of the other lyersD wwh silly ts s rndomized version of the mximiztion of p(x s |y) on every pixelF gomputtionllyD the numer of itertions of wwh tht su0es to reh onvergene is usully signi(ntly smller thn the numer of individul opertions leding to the mximiztion of p(x s |y) on every pixel with respet to the lss lelF eordinglyD wwh is expeted to e dvnE tgeous from omputtionl viewpointF his is onsistent with vrious previous works using ww on hierrhil wp models @seeD eFgFD érez et lFD PHHH tht omines ww nd sgwY nd oisin et lFD PHIR for ww nd wwhAF pigure SFPX wultiEdte ww estimtion on the qudEtreeX aP nd two dtesF

= m) = K mnt i=1 π mnt i F mnt i (y s |θ mnt i ), ∀s ∈ S n t @SFIPA
where π mnt i re the mixing proportionsD θ mnt i is the set of the prmeters of the i th hp mixture omponent of lss m t sle level n nd time tD nd F mnt i is the orresponding prmetri fmily (n = 0, 1, . . . , R; m = 0, 1, . . . , M -1; t = 0, 1)F hen the dt t sle level n nd time t is n optil imgeD the lssEonditionl mrginl hp p mnt (y|m) relted to eh lss m is modeled y multivrite qussin mixture pigueiredo nd tinD PHHP with set of prmeters θ mnt i = {µ mnt i , Σ mnt i } ssoited with the orresponding mens nd ovrine mtriesF pormlly the omponent i of the multivrite qussin mixture is given y @for ese of nottion the susripts m, nD nd t re droppedAX

p i (y|m) = (2π) -d 2 |Σ i | -1 2 exp [- 1 2 (y -µ i ) T Σ -1 i (y -µ i )] @SFIQA
his qussin ssumptionD espeilly when omined with (nite mixture is wellEknown nd widely epted model for the sttistis of optil dtF hnks to the linerity of the wvelet opertorD the sme ssumption lso holds for the resulting trnsformed levels of the qudEtreeF he use of (nite mixtures insted of single hps o'ers the possiility to onsider heterogeneous hpsD usully re)eting the ontriuE tions of di'erent mterils present in eh lssF his lss heterogeneity is relevnt when we ddress r imgesF he prmeters of the mixture model θ mnt i in the qussin mixture re estimted through the stohsti expettion mximiztion @iwA lgorithm geleux et lFD IWWTD whih is n itertive stohsti prmeter estimtion lgorithm developed for prolems hrterized y dt inompleteness nd pprohingD under suitle ssumptionsD mximum likelihood estimtesF por eh sle nd timeD iw is seprtely pplied to the trining smples of eh lss to estimte the relted prmeters s shown in pigure SFQ using the lgorithm SFQFIF For each class m ∈ {0, 1, . . . , M }, time t ∈ {0, 1}, and scale n ∈ {0, 1, . . . , R} and at each iteration j ∈ [1, j max ], dropping the subscripts m, n, and t for ease of notation: E-step:

For each observation y k , k ∈ [1, N ] and the i th mixture component, i ∈ [1, K], posterior probability estimation:

τ j i (y k ) = π j i p(y k |µ j i , Σ j i ) K l=1 π j l p(y k |µ j l , Σ j l )
; S-step:

A label s j (y k ) is associated to each training y k of the considered class by using the previous posterior estimation

{τ j i (y k ) : i ∈ [1, K]};
M-step:

For each mixture component i, the proportions and the parameters at the iteration j + 1 are given by:

π j+1 i = Q j i N µ j+1 i = 1 Q j i N k=1 yτ j i (y k ) Σ j+1 i = 1 Q j i N k=1 τ j i (y k )(y -µ j+1 i )(y -µ j+1 i ) T
where Q j i is the set of samples assigned to the i th mixture component in the S-step.

Transition probabilities

he trnsition proilities etween onseutive sles nd onseutive dtes deterE mine the properties of the hierrhil wp euse they formlize the uslity of the sttistil intertions involvedF hereforeD they must e refully de(nedF sn the proposed methodD two types of proilities involve timeX (i) he pirst is the set of temporl trnsition proilities t the identil sle p(x s -|x s = )D whih re estimted using spei( formultion of the expettionEmximiztion @iwA lgorithm pigueiredo nd tinD PHHPF en itertive (xedEpoint iwElike lgorithm is performed to estimte the prior joint proilities p(x s -, x s = ) for eh sle nD nd the temporl trnsiE tion proilities re then derived fruzzone et lFD IWWWF he proilities p(x s -= m, x s = = m ) D where m nd m rnge in Λ = {0, 1, . . . , M -1}D re regrded s the elements of n M × M mtrix JD whih is omputed y mximizing the following pseudoElikelihood (n = 0, 1, . . . , R)X

L(J) = s∈S n 1 x s-x s = p(x s -, x s = )p(y s -, y s = |x s -, x s = ) @SFIRA
he reursive eqution to e used to mximize the pseudoElikelihood in @SFIRA is the followingX

p k+1 (x s -, x s = ) ∝ s∈S n 1 p k (x s -, x s = )p(y s -|x s -)p(y s = |x s = ) x s -x s = p k (x s -, x s = )p(y s -|x s -)p(y s = |x s = ) @SFISA
where p k (x s -, x s = ) is the itertive joint proility estimte t the k th iw itertionD nd N is the totl numer of pixelsF hese estimtes re initilized y ssigning equl proilities to eh pir of lssesX

p 0 (x s -, x s = ) = 1 M 2 @SFITA
(ii) he seond type of trnsition proilities tht involve time is the hildEprent trnsition proility p(x s |x s -, x s = ) F o our knowledgeD seEspei( formuE ltion of iw is not ville for interEsle trnsition proilitiesF roweverD prmetrilly modeling these proilities hve demonstrted n e'etive hoie in the se of singleEdte lssi(tion s it yielded urte results to e otined vferté et lFD PHHHD oisin et lFD PHIPF sndeedD we extend here the lgorithm proposed in fum et lFD IWUHD whih fvors the identity eE tween the hildren nd prents @in the urrent nd previous dtesAD ll other trnsitions eing unlikelyX

p(x s |x s -) =                  θ x s = x s -= x s = ϕ ((x s = x s -) or (x s = x s -)) and x s = = x s - 1 -θ M -1 x s = x s -and x s = x s = and x s -= x s = 1 -2ϕ M -2 x s = x s -and x s = x s = and x s -= x s = @SFIUA
with the prmeters θ > 1 M nd 1 M < ϕ < 1/2F rereD θ hs the sme mening s in eqution @SFTAD nd the sme prmeter vlue is used in oth trnsition proilitiesF pigure SFSX ixmples of imges time series quired over ortEuErine @ritiA y qeoiye stellite qeoiyeF pive lnd over lsses hve een onsidered for oth dt setsX urn @redAD wter @lueAD vegettion @greenAD re soil @yellowAD nd ontiners @purpleAF e note tht these lsses represent semntilly high level lnd oversF roweverD lssi(tion mp ssoited with more detiled lsses n e produed when sophistited ground truth is villeF sn the present workD mnully nnotted nonEoverlpping trining nd test sets were seleted in homogeneous res @s shown in pigure SFT sn the se of the léides imgesD the (nest resolution of the multiEresolution pyrmid @level HA ws set equl to the (nest resolution of the input pnhromti imges @iFeFD HFS mAF goEregistered multiEspetrl imges @t P mA were integrted in level P of the pyrmidF o (ll level I of eh qudEtreeD wvelet deomposition of eh pnhromti imge ws usedF es preliminry experimentD the omintion of the proposed method with numerous wvelet opertorsD inluding huehiesD iorthogonlD nd reverse iorthogonl wveletsD symletsD nd oi)ets of vrious orE ders wlltD PHHVD ws exminedF he results were similrD nd the min di'erene relied on the level of smoothness of the (nl lssi(tion mpF yn one hndD s shown in pigure SFUD the verge of the overll uries otined on the test sets of ll individul dtes ws remrkly stle s funtion of the seletion of the wvelet opertorD suggesting tht this seletion is not ritil in the pplition of the proposed pprohF yn the other hndD n exeption ws represented y the huehies wvelets of order IH @dIHA whose omintion with the proposed multiE resolution method resulted in higher uries thn the other onsidered wvelet trnsformsF his wvelet opertor will e used in ll other experiments disussed in this disserttion @see pigure SFUAF pigure SFUX everge of the overll uries otined on the test sets of ll indiE vidul dtes results using severl wvelet fmiliesF he qeoiye imge resolution is HFS meter @the (nest resolution of the multiE resolution pyrmidAF gomments similr to those reported for the experiments with léides imges regrding the seletion of the wvelet opertor hold here s wellD nd dIH ws used to (ll level I nd level P of the pyrmidF es disussed in etion SFPD the proposed method depends on four prmetersD iFeFD β in @fFWAD θ in @SFUA nd @SFIUAD ϕ in @SFIUAD nd RF the lssi(tion results inluded in this mnusript were otined using the following prmeter vluesX β = 0.8D θ = 0.85D ϕ = 0.48D nd R = 2D iFeFD three levels in eh qudEtreeF snluding more deomposition levels generlly helps in disriminting homogeneous lnd overs ut might result in the removl of smllEsize imge detils @see pigure RFRAF sndeedD with R = 2D the lssi(tion mp is generted t 50Em sptil resolution for oth dt setsD while the orsest sle orresponds to 2Em resolutionF snluding orserE resolution fetures would generlly fvor sptil smoothness ut my progressively hinder the pility to disriminte lsses hrterized y sptil detils suh s 4uildings4 nd 4ontinersF4 he vlue of β ws utomtilly optimized y pplying the wellEknown pseudoElikelihood method fesgD IWUR to the trining smplesF eordinglyD userGopertor does not hve to perform trilEndEerror proedure to set βF sn generlD lower vlue of β prevents sptil overEsmoothing t the prie of n ury derese on test smples loted inside homogeneous regions ssoited with the sme themti lssF es β is utomtilly optimizedD the only free prmeters re θ nd ϕF eording to @SFIUAD θ is the proility tht siteD its prent t the sme dte nd its prent t the previous dte shre the sme lss lelY ϕ is the proility tht site shres the sme lel of one of the two prents while the prents disgreeF por @SFIUA to de(ne proility distriutionD θ nd ϕ n tke vlues in [0, 1] nd [0, 0.5]D respetivelyD in the se of M = 5 lssesF pigures SFV nd SFW show the ehvior of the overll ury @yeA of the proposed method on the test set s eh one of these two prmeters rnge in these intervls while the other prmeter is (xed to the forementioned referene vlues @iFeFD θ = 0.85 nd ϕ = 0.48AF pigure SFWX hetils of the the lssi(tion mps otined y the proposed method when pplied to the léides dt set with di'erent vlues ϕ nd θF yn one hndD these plots suggest tht the method is sensitive to the vlues of θ nd ϕF his is n expeted result euse they involve the uslity of the modelF yn the other hndD limited sensitivity ws oserved nd ye remined higher thn pproximtely 85% s long s ϕ nd θ were lrger thn 0.4 nd 0.85D respetivelyF yn the ontrryD it ws silly for reltively extreme nd not meningful vlues of θ or ϕ tht poor vlues of ye were otinedF por exmpleD ϕ = 0.2 implies tht there is 15% proility tht the lels of site nd of its two prents re ll di'erent @see @SFIUA with M = 5AF feuse of the very high interEsle nd temporl orreltions ssoited with the multiEresolution dt setsD this event is highly unlikelyD nd ϕ = 0.2 yields to signi(ntly overestimting its hnes of ourring in the lssi(tion proessD thus 'eting yeF imilrlyD θ = 0.5 implies 12.5% proility tht the prents of site t the two dtes gree on ertin lss memership ut the site disgreesD nother outome tht is very unlikely euse of interEsle orreltion nd whose proility is overestimtedF hereforeD lthough the proposed method is overll sensitive to θ nd ϕD the experimentl nlysis suggests tht the mening of the two prmeters in reltion to interEsle nd temporl orreltion llows userGopertor to rther esily determine vlues of or rnges on these prmeters tht led to urte lssi(tion mpsF reliminry experiments lso pointed out tht the use of the wwh optimiztion tehnique resulted in signi(nt redution in the numer of itertions needed to estimte the lss lel of eh pixel s ompred to the diret mximiztion of the posterior mrginlsF o reh onvergene in the se of one single level of qudEtree inluding 1600 × 1000 pixels nd 5 lssesD wwh required fewer thn 1000 itertionsF e rell thtD in eh wwh itertionD one individul pixel is exminedF yn the ontrryD the diret hoie of the lss lel tht mximizes the posterior mrginl on eh pixel required to iterte over ll 1600 × 1000 pixels nd 5 lssesD thus tking muh longer timeF he results otined y the proposed method were ompred to those generted yX (i) the tehnique in vferté et lFD PHHHD used s multiEresolution singleEdte enhmrk in oth its (a) wwE nd (b) weEsed formultionsY (ii) the wpEsed lgorithm proposed in welgni nd erpioD PHHQD used s singleEresolution multiEtemporl enhmrkY (iii) ontextul omintion of the K nerest neighor nd wpEsed pE prohesD used s singleEresolution singleEdte enhmrkY (iv) the wellEknown KEmens lustering tehniqueD used s si unsupervised enhmrkF he results of the proposed nd previous tehniques re reported in the following susetion long with further detils on their pplitionsF

Experimental Results and comparisons

sn this setionD we present the lssi(tion mps nd disuss the orresponding lssi(tion uries tht were otined on the test setF pigure SFII nd E le SFP refer to the results otined using léides imgesD nd pigure SFIP nd le SFQ regrd those otined from the qeoiye quisitionsF ell omputtion times reported in the tles refer to gCC implementtion on n sntel iU qudE ore @PFRH qrzA VEqfEew TREit vinux systemF he nlysis of the lssi(tion mps hs suggested tht the proposed hierrhil method leds to urte reE sultsF sn prtiulrD severl experimentl omprisons were performed with methods exploiting multiE or singleEresolutionD multiE or singleEdteD supervised or unsuperE vised pprohesF pirstD the results of the proposed tehnique were ompred to the seprte hierrhil lssi(tion results otined t individul dtes using the multiEresolution singleEtime method in vferté et lFD PHHHD in oth its ww @pigE ure SFII@A nd pigure SFIP@AA nd we @see pigure SFII@dA nd pigure SFIP@dAA formultions nd using Q level pyrmid with the following prmetersX β = 0.8 nd θ = 0.85F e rell tht severl extensions of the method in vferté et lFD PHHH hve een developed inluding the pproh presented in oisin et lFD PHIP for the spei( se of multiEsensor lssi(tion nd sed on the integrtion of the hierE rhil wp model of vferté et lFD PHHH with opul funtions for merging dt from oth optil nd e sensors within the sme pyrmidF sn the present thesisD the fous is on multiEtemporl lssi(tion with optil imges nd not on multiE sensor fusionF eordinglyD we used the originl method in vferté et lFD PHHH for omprison purposesF he results of the omprison show the e'etiveness of the proposed multiEtemporl hierrhil model in fusing the temporlD sptilD nd multiEresolution informtion ssoited with the input dt @see le ??AF sn prE tieD the use of one qudEtree struture with the ww riterion yields 4loky4 segmenttion @see pigure SFIH @AAF his phenomenon n e explined y the ft tht two neighoring sites t given sle my not hve the sme prentF sn this seD oundry ppers more esily thn when they re linked y prent nodeF hese loky rtifts re voided y the use of the multiEtemporl hierrhil struture proposed in this thesis in whih usl reltionships etween prents nd o'spring in the sme qudEtree re relxed y the introdution of other usl reE ltionships over time nd sle @see pigure SFIH@AAF yne of the min soures of mislssi(tion in the singleEdte results is the onfusion etween the 4urn4 nd 4vegettion4 lssesF his mislssi(tion is redued in the multiEtemporl lssiE (tion otined y the proposed method euse of the modeling of the temporl reltionships mong the input multiEresolution dtF purthermoreD s expetedD the we riterion ws poorly e'etive when pplied to the onsidered hierrhil struture euse errors were propgted from the root to the leves nd led to seE vere mislssi(tionsD espeilly regrding the lsses tht most strongly overlp in the feture spe @eFgFD 4urn4 nd 4ontiners4Y see pigures SFII@dA nd SFIP@dAAF (a) (b) pigure SFIHX @A floky rtifts otined using qudEtree struture vi ww formultionD @A redution of these loky rtifts using the proposed methodF eondD in the ontext of multiEtemporl lssi(tionD the proposed lssi(er ws ompred to the multiEtemporl singleEresolution wpEsed method proposed in welgni nd erpioD PHHQF st uses the mutul pproh nd onsists in perE forming idiretionl exhnge of the temporl informtion etween the @nonE hierrhilA singleEtime wp models ssoited with onseutive imges in the seE queneF sn the form of n pproprite energy funtionD eh singleEtime wp model integrtes three types of informtion @spetrlD sptil ontextulD nd temporl onE textulA using multilyer pereptron @wvA neurl network to extrt the spetrl informtionF he results reported in le SFQ show tht etter exploittion of the sptioEtemporl informtion llowed the proposed sde multiEresolution pE proh to provide more urte results thn the previous mutul singleEresolution pproh in welgni nd erpioD PHHQF wore generllyD the mutul pproh reE dues the risk of propgting the lssi(tion error etween onseutive dtesD while the use of the hierrhil shem provided more urte lssi(tion mpsD t lestD on the onsidered dt setsF purthermoreD euse of the hierrhil sE pets nd the nonEitertive lgorithmD only few minutes were neessry to otin stisftory results using the proposed pproh ompred to those otined y the mutul pproh tht required muh longer omputtion time @severl hoursAF eE ording to the formultion of the method in welgni nd erpioD PHHQD this time inluded the times required to ompute the texture fetures from the given imge time seriesD to trin nd pply n wv neurl network for the imge of eh dte using the kEpropgtion lgorithmD nd to estimte the prmeters of the orE responding wp model using the seEspei( prmeter optimiztion proedure in sáñez nd imóD PHHQF he lssi(tion mps otined using the wellEknown uEnerestEneighors @uExxA method re lso shown in pigures SFII@hA nd SFIP@hAF uExx ws used s enhmrk nonEprmetri lssi(erF st is nonEontextulD so to perform fir omprison etween the proposed method nd sptilEontextul tehniqueD it ws omined with n wp modelF e hidden wp whose unry term ws expressed in terms of the pixelwise posterior proilities estimted y uExx nd whose ontextul term ws represented y n isotropi otts model ws usedF u a QH ws estimted y ross vlidtion on the trining setD nd the smoothE ing prmeter of the otts model ws optimized using the utomti method in erpio nd woserD PHHTD whih is sed on the roEushyp lgorithmF he nuE meril results on the test sets suggest tht this singleEsle wpEsed method @see les SFP nd SFQA leds to rther poor ury nd severe sptil oversmoothing s shown in pigures SFII@gA nd SFIP@gAF his is onsistent with the ft tht this omined uExx C wp lssi(er is intrinsilly singleEresolution nd singleEdteD nd n exploit neither the multiEresolution nor the multiEtemporl struture of the input dt setF sn the mp in pigure SFIP@gAD otined from qeoiye dtD the omE ined uExx C wp well disriminted the 4wter4 nd 4urn4 lsses ut lmost did not identify the other themti lsses due to the strong spetrl overlpping nd the imlne etween the trining smple sizes of these lssesF pinllyD further omprison ws performed etween the results of the proposed method nd those of n unsupervised lgorithmF uEmens ws used for this enhmrk omE prison s wellEknown onsolidted pprohD nd ws pplied with K = 5F his numer of lusters ws used to mth the numer of lsses in eh dt setF he lusters otined y uEmens generlly do not oinide with the themti lsses of supervised lssi(tion prolemF en lternte strtegy ould e toD (rstD pE ply uEmens using signi(ntly lrger numer of lustersD nd thenD perform lusterEtoElss ssignment either mnully or on the sis of the trining setF sn either seD this ssignment would inorporte prior knowledgeF his experiment ws ment s enhmrk omprison with n unsupervised method using no prior knowledgeF eordinglyD the simple hoie K = 5 ws eptedF es expeted due to its unsupervisedD nonEontextulD nd singleEresolution formultionD uEmens perE formed the worst in terms of lssi(tion uryD while it exhiited the lowest omputtion time @see pigures SFII@hAD SFIP@hAAF le SFQX glssi(tion uries on the test set of the qeoiye dtsetX lss uriesD overll uryD nd omputtion timeF ixperiments were onduted using one @ITHHxVHHA imge t level HD one @VHHxRHHA imge t level I nd one @RHHxPHHA nds t level P on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF sn the proposed methodD multiEdte nd multiEresolution fusion is sed on exE pliit sttistil modelingF he method omines joint sttistil model of the onsidered input optil imges through hierrhil wrkov rndom (eld modelingD leding to sttistil supervised lssi(tion pprohF e hve developed novel wwEsed hierrhil wrkov rndom (eld model tht onsiders multiEtemporl informtion ndD thusD supports the joint supervised lssi(tion of multiple imE ges tken over the sme re t di'erent times nd di'erent sptil resolutionsF e nlyzed the results otined with the proposed method through experiments with multiEtemporl léides nd qeoiye dt setsF he experimentl results show tht the method is le to provide urte lssi(tion mpsF he proposed lgorithm ws ompred to previous singleEdte multiEresolution method nd previous multiEdte singleEresolution oneD oth sed on wp models ssoited with suitE le @hierrhil or singleEsleA pixel lttiesD nd ouple of wellEknown lssi(ers inluding ontextul omintion of the u nerest neighor nd wpEsed pE prohesD used s singleEresolution singleEdte enhmrkY nd the uEmens lusE tering tehniqueD used s si unsupervised enhmrkF he proposed tehnique hs een dvntgeous in terms of the lssi(tion ury on the test setD the spE til regulrity of the lssi(tion mpsD the minimiztion of sptil rtiftsD nd the trdeo' with respet to omputtion timeF hese results show the e'etiveness of the lgorithm in fusing oth multiEtemporl nd multiEresolution informtion for supervised lssi(tion purposes nd on(rm tht wp models represent powerE ful fusion tool in remote sensingF he omputtionl dvntges of the hierrhil wpsD for whih ext reursive formultions of the ww deision rule re fesile with no need for timeEexpensive wetropolis or qis smpling proeduresD hs lso een on(rmed y the experimentl results of the proposed method nd those of the enhmrk singleEresolution or multiEresolution lssi(ersF he proposed method is sed on n wp model on seEspei( topology tht omprises multiple hierE rhil qudEtreesD eh ssoited with n quisition dteF velet trnsforms re used to (ll in those levels of eh qudEtree tht re ssoited with input reE mote sensing imgeryF he seletion of the wvelet opertor mong lrge fmily of possile trnsforms ws not ritil euse most trnsforms led to lssi(E tion results with similr uriesF xeverthelessD huehies wvelets of order IH yielded higher uries thn the other onsidered trnsformsF yne min dvnE tge of the proposed lssi(er is tht it n e extended to e used for optil dtD syntheti perture rdr @eFgFD gywyEkywedD or eheeEPA or multiEsensor dtF he extension to the multiEsensor se will e disussed in the next hpterF 6.1 Multi-sensor PDF modeling es disussed in ghpter ID the vilility of di'erent kind of sensors is very dE vntgeous for lnd over mpping pplitionsF st llows us to pture wide vriety of properties of the ojets ontined in sene s mesured y eh sensorF hese properties n e exploited to extrt riher informtion out the imged reF sn prtiulrD s previously disussedD the opportunity of joint vilility of e nd optil imges n possily o'er high resolutionD llEwetherD dyGnightD short revisit time dtD polrimetriD ndD multiEfrequeny quisition pilitiesF his potentil is espeilly emphsized y urrent nd forthoming stellite missions for iyY eFgFD entinelD léidesD qeoiyeD gywyEkywed @guAD eheeEP @PA nd erreED whih onvey huge potentil for multiEsensor optil nd e oservtionF hey llow sptilly distriuted nd temporlly repetitive view of the monitored re t the multiple sptil slesF roweverD the use of multiEsoure imge nlysis for lnd over lssi(tion purposes hs een mostly ddressed so fr y fousing on singleEresolution multiEsensor optilEe imgery wheres the joint use of multiEsensor nd multiEresolution pilities hs een more srely investigtedF e generl disussion of the lssil literture on multiEsensor optilE e imge lssi(tion hs een presented in etion PFPF his pproh ers the ovious dvntge of simpliity ut isD in generlD suoptimlF prom methodologE il viewpointD when multiEsensor @optil nd eA or multiEresolution imges of given sene re villeD using them seprtely disrds prt of the orreltions mong these multiple dt soures ndD most importntlyD their omplementrityF es illustrted in pigure TFID e nd multiEspetrl imges exhiit omplementry pigure TFIX ensitivity to loud over nd ojet size using di'erent wvelength rngeF properties in terms of wvelength rnge @tive mirowve vsF pssive visile nd infrredAD noisy ehvior @often strong in e due to spekleD usully less ritil in optil imgeryAD fesiility of photoEinterprettion @usully esier with optil thn with e dtAD impt of tmospheri onditions nd loud over @strong for optiE l quisitionsD nd lmost negligile for eA nd sensitivity to unEillumintion @strong for optil imgery nd negligile for eAF his mkes the joint use of r optil nd e imgery prtiulrly interesting for mny pplitions relted to environmentl monitoring nd risk mngementF ithin this frmeworkD there is de(nite need for lssi(tion methods tht uE tomtilly orrelte di'erent sets of imges tken on the sme re from di'erent sensors nd t di'erent resolutionsF yne wy to ddress this prolem is to reE sort to n expliit sttistil modeling y (nding joint proility distriution given the lssEonditionl mrginl hps of the dt olleted y eh sensor @see pigure TFPAF he joint sttistis n e designed y resorting to metEqussin disE triutions torvik et lFD PHHWD multivrite sttistis suh s multivrite opuls oisin et lFD PHIRD or nonEprmetri density estimtors pukungD PHIQF rowE everD employing heterogeneous dt @eEoptil in our seA mkes the tsk of (nding n pproprite multivrite sttistil model omplexD time demndingD nd possily prone to overE(ttingF pigure TFPX wultivrite sttistil modelingF sn this hpterD two methods re proposed for the joint supervised lssi(tion of multiEsensor imges inluding e nd optil omponentsD quired t multiE ple sptil resolutionsF he rtionle of oth pprohes is to tke ene(t of the dt fusion pilities of hierrhil wps nd to void the omputtion of joint sttistisF e novel pprohD sed on multiple qudEtrees in sdeD pplied to multiEsensor nd multiEresolution fusion is desriedF sn the (rst proposed methodD for eh sensorD the input imge is ssoited with seprte qudEtree struture on the sis of its resolutionF he gol is to jointly lssify e nd n optil imge quired over the sme reF he proposed pproh formlizes supervised fyesin lssi(er within this multiple qudEtree topology tht omines lssE onditionl sttistil model for pixelEwise informtion nd hierrhil wp for multiEsensor nd multiEresolution ontextul informtionF he seond proposed method fouses on the spei( se of the fusion of multiE frequeny e dt olleted y the gywyEkywed @ ndA nd eheeEP @g ndA sensors together with optil léides dtF e multiple qudEtree struE ture is used gin ut optil nd e dt re jointly inluded in eh qudEtree to tke ene(t of the (nest resolution ville from eh sensorF gompred to the (rst proposed methodD whih onsiders the fusion of dt from generlly ritrry e nd optil sensorsD this seond method fouses on spei( omintion of speEorne e nd optil sensors to investigte the synergy mong the multiE frequeny nd multiEresolution informtion they provideF qiven trining set for eh input dtD for eh lss mD sle n nd sensorD we model the orresponding lssEonditionl mrginl using (nite mixtures of indeE pendent gry level distriutions s detiled in etion SFQFIF he mixture modeling is performed depending on the di'erent types of remote sensing imgery used in the studyF sndeedD when the input dt t the n th sle level is n optil imgeD lssEonditionl mrginl hp relted to eh lss m n e modeled y qusE sin mixture modelF yn the oppositeD e quisitions re known to e 'eted y spekleF por this resonD we use pproprite e spei( models for suh imgesD suh s the generlized qmm distriution vi et lFD PHIIX where ν mn , σ mn nd κ mn re the relted prmeters nd Γ(.) is the qmm funtionF e wide vriety of hp fmilies n e dedued from @TFIA y vrying the prmetersF his inludesX yleigh (ν mn = 2, κ mn = 1)D xkgmi (ν mn = 1)D logEnorml (κ mn -→ ∞) nd (κ mn = 1)F he evolution of the proility density of the generlized qmm distriution s funtion of eh individul vrileD the two other eing (xed is given in pigure TFQF he prmeters of the mixture model for oth e nd optil imges re estimted through the iw lgorithm @see elgorithm SFQFIAF 6.2 The rst proposed multi-sensor hierarchical model 6.2.1 methodology sn this setionD the (rst proposed method for eEoptil multiEresolution lssi(E tion sed on hierrhil wrkovin model is presentedF vet n optil imge nd e imgeD quired t the sme time over the sme reD e givenD together with trining mp hrterizing M themti lssesF he multiEtemporl struture developed in the previous hpter @see etion SFIA is dpted in order to hndle the heterogeneous dt quired y di'erent sensors with di'erent physil propertiesF he novelty in the proposed pproh is the use of multiple qudEtrees in sde pigure TFQX he evolution of the proility density of the generlized qmm distriution s funtion of σ mn @AD ν mn @AD nd κ mn @AF @see pigure TFRAD eh ssoited with the set of imges given y one spei( senE sorF sn this respetD this method tkes ene(t of the pility of multiEqudEtree struture to fuse the informtion ssoited with orrelted dt souresD pilE ity tht hs een exploited for multiEtemporl imge lssi(tion in the previous hpter nd is investigted in the multiEsensor se hereF his pproh lso ims t exploiting the multiEsle informtion tht is typilly ssoited with either e or optil r imgery y using hierrhil wpF st is worth relling thtD here s in the se of multiEtemporl imgesD the proposed hierrhil struture implies onstrint mong the resolutions of the input multiE sensor imges t the vrious levels of the pyrmidF he hoie of qudEtree topology imposes ftor of P etween the sptil resolutions of the imges t two suessive levelsF he 4empty4 levels re gin (lled in through wvelet deomposition of the imges ssoited with the (ner levelsF he qudEtree struture llows in very nturl wy the use of n expliit stE tistil modeling through hierrhil wrkov rndom (eld formultion using series of rndom (elds t vrying slesD resolutionsD nd sensorsD on the sis of the trnsitions de(ned on the qudEtrees s shown in pigure TFRF eordinglyD the formultion of ww developed in etion SFP is used through multiple qudEtrees in sdeD while here eh qudEtree is iunivolly ssoited pigure TFRX wultiEsensor hierrhil strutureF with the dt quired y one individul sensorF pei(llyD the posterior mrginl p(x s |y) of the lel of eh site s in the qudEtree relted to the optil sensor is expressed s funtion not only of the posterior mrginl of the prent node p(x s -|y) in the sme qudEtree ut lso of the posterior mrginl of the prent node in the qudEtree ssoited with e imges p(x s = |y) s shown in @TFPAD with the im to hrterize the eEoptil orreltions ssoitedD t di'erent slesD with distint imges in the input multiEsoure dt iFeFX

p(x s |y) = x s -,x s = p(x s , x s -, x s = |y d(s) ) xs p(x s , x s -, x s = |y d(s) ) p(x s -|y)p(x s = |y) @TFPA
o ompute these proilitiesD we tke ene(t from the hierrhil struture de(ned ove nd we use two reursive psses on the qudEtreeD referred to s 4ottomEup4 nd 4topEdown4 psses s sustntited in elgorithm TFPFIF Algorithm 6.2.1 (Multi-sensor hierarchical technique)

Quad-tree associated with SAR images

The classication is done at the rst quad-tree composed by the SAR image and of the corresponding wavelet transforms, using the classical MPM using the Algorithm 4.2.1 Quad-tree associated with optical images Initialization, (s ∈ S R opt ):

To estimate the prior in the root given the resulting classication map, we use a spatial Markovian model which takes into account the contextual information given by SAR images (as detailed in the case of multi-temporal, see Section 5.2.1) in the estimation of the prior probability distribution associated with the quad-tree of the optical images.

⇓ Top-down pass I, (s ∈ S n opt , n = 0, 1, . . . , R -1): ⇓ Top-down pass II, (s ∈ S n opt , n = 0, 1, . . . , R -1):

p(x s ) = x s - p(x s |x s -) p(x s -) Initialization, (s ∈ S 0
p(x s |y) = x s -,x s = p(xs,x s -,x s = |y d(s) ) xs p(xs,x s -,x s = |y d(s) ) p(x s -|y)p(x s = |y)

Experimental results

ixperiments re disussed regrding two dtsets olleted over ortEuErine @ritiA inludingX (i) e single polriztion singleElook gu imgeD PHIH @ esY pigure TFS@AA with rr polriztionD stripmp quisition modlityD nd PFSEm pixel spingY (ii) oregistered qeoiye qf quisitionD PHIH @ qeoiyeY pigure TFS@A@AAF (a) (b) pigure TFSX @A e imge @ esAD @A one hnnel from the optil imge @ qeoiyeA wultiEsle fetures re extrted through Ph disrete wvelet trnsformF es in the se of multiEtemporl imge lssi(tionD preliminry experiments suggested huehies wvelets of order IH s n pproprite hoieF pive lnd over lsses hve een onsideredX urn @redAD wter @lueAD vegettion @greenAD soil @yellowA nd ontiners @purpleAF e ompre the proposed method with the lssi(tion results otined using the following tehniquesX (1) eprte hierrhil lssi(tions of the imges provided y the two sensors @pigure TFT@A for optil nd pigure TFT@A for eAF sn this seD the lssiE (tion is otined reursively over the sles y omining the pproh in vferté et lFD PHHH with the sme hp estimtion lgorithms desried in etion SFQFI for optil nd e dtF e rell tht the use of qudEtree struture in ww sheme my yield 4lokly4 segmenttions @detils n e found in vferté et lFD PHHH nd etion RFPAF

(2) he multiEsensor multisle method proposed in oisin et lFD PHIRD in whih model for the multivrite joint lssEonditionl sttistis of the oEregistered input imges t eh resolution is designed y resorting to multivrite opuE lsF he estimted joint proility density funtion is plugged into hierrE hil wrkovin model sed on qudEtree struture @see pigure TFT@dAAF (3) he multisensor singleEsle pproh in torvik et lFD PHHW @see pigE ure TFT@eAAD in whih the likelihood term is onstruted y merging generlized qmm @for eA nd qussin @for optilA mrginls into metEqussin distriutionF he lssi(tion is otined y the mximum likelihood ruleF

(a) (b) (c) (d) (e) (f)
pigure TFTX @A ground truth mp @inluding trining nd test resAD @A hierrE hil wpEsed lssi(tion otined for the optil imgeD using the method in vferté et lFD PHHHD @A hierrhil wpEsed lssi(tion otined for the e imgeD using method in vferté et lFD PHHH with generlized qmm lssE onditionl hpsD @dA result of the multiEsensor nd multiEresolution method in oisin et lFD PHIRD @eA result of the multiEsensor method in torvik et lFD PHHWD @fA hierrhil wpEsed lssi(tion otined y the proposed multiEsensor nd le TFIX glssi(tion uries for the ortEuErine dtsetF ixperiments were onduted on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF he resulting lssi(tion mps show tht the proposed hierrhil method leds to urte resultsD espeilly s ompred to seprte hierrhil lssi(tions of the imges provided y the two sensors @see pigures TFT@AD@AAF sndeedD experimenE tl results otined y using only e dt urtely detet rods nd ontinersD while the results generted y using only optil dt etter disriminte lsses tht re sptilly homogenousF he proposed method e'etively tkes ene(t from oth e nd optil imgeryD nd llows generting lssi(tion result tht viE sully well disrimintes ll lsses in the onsidered r dt setF st is interesting to notie tht the detetion of the 4ontiners4 lssD whih is generlly very overlpE ping with other urn or uiltEup lsses in the feture speD improves signi(ntly ompred to the method in oisin et lFD PHIRD in whih the lssi(tion mp is over smoothed @see pigure TFT@dAAF gompred to the previous multiEsensor lssiE (tion method in torvik et lFD PHHW @see pigure TFT@eA nd le TFIAD whih is sed on trnsformtions of the input fetures to ommon jointly qussin doE minD the proposed lgorithm provides sptilly more regulr lssi(tion resultD thnks to ontextul wp modeling nd wvelet feture extrtionF here is n improvement using the proposed method ompred to the hierrhil multiEsensor method in oisin et lFD PHIR @see pigure TFT@dAA nd le TFIA in terms of uE ry nd omputtion timeF por this methodD the min soure of mislssi(tion is the ontiner reD where sphlt is erroneously lssi(ed s vegettionF woreE overD with the proposed methodD we void the omputtion of joint hpsD while opuls re used in torvik et lFD PHHW nd metEqussin distriutions re used in oisin et lFD PHIRF his hoie results in reduing the omputtion time @see le TFIAF 6.3 The second proposed multi-sensor hierarchical model 6.3.1 Methodology he seond proposed method ddresses spei( prolem of multiEsensor optilE e fusion foused on exploiting the synergy etween urrent spe missions for iy rrying r optil nd e pylodsF he joint use of gywyEkywedD eheeEPD nd léides imges is investigted with the im of tking ene(t from the relted multiEfrequeny informtion @rdr ndD rdr g ndD optil xs ndAF he interest of this spei( study is further enfored y the role of these sensors in the re of r remote sensingF ell the three sensors support multiple sptil resolutions up to HFS m for léidesD pproximtely 1 m for gywyEkywed @in its spotlight modeAD nd round 1.5 m of pixel sping for eheeEP @in its ultr(ne modeAF o oth ene(t from the multiEsensor nd multiEfrequeny informtion they onvey nd ensure the pility of mpping lnd over t the (nest of the ville sptil resolutionsD seEspei( hierrhil wp sed on doule qudEtree is proposed he pproh extends the reent method proposed in the previous hpter nd setionD whih foused on multiEresolutionGmultiEtemporl imge lssi(tion to multiEsle nd multiEsensor model tht fuses the sptilD multiEresolutionD multiEsensorD nd multiEfrequeny informtion onveyed y input imges olleted y guD PD nd one optil sensor @léidesA t multiple sptil resolutionsF por eh of the two e sensorsD the input imges re inserted in seprte qudEtree on the sis of their sptil resolutionsF hese resolutions re expeted to e orser thn the (nest resolution tht n e hieved using léides dtF hereforeD the input léides imge is emedded in the nestEresolution levels @iFeFD the levesA of oth qudEtreesF impty levels of eh qudEtree re (lled in using wvelet trnsforms of the léides dt @see pigure TFUAF henD elgorithm TFPFI is pplied to ompute reursively the posterior mrginl of eh siteD while qussin nd generlized qmm hps re used to estimte the lssEonditionl sttistis of the optil nd e hnnelsD respetively @nd of the wvelet fetures derived from themAF pigure TFUX wultiEsensor hierrhil struture @se PAF

Experimental results

sn this setionD we disuss the results of the experimentl vlidtion of the developed model on two dtsets quired over ortEuErine @ritiA usingX (i) pnhromti léides quisition t HFSm resolution @léidesD gxi disE triution eirus hD PHIIAD shown in pigures TFV@A nd TFW@AF (ii) gu imge @ esD PHIIAD ndD rr polriztionD potlight mode @Im resolutionAD geoodedD singlelookD shown in pigures TFV@A nd TFW@AF (a) (b) (c) pigure TFVX pirst site used for experimentsX @A léides @ gxi distriution eirus hAD @A gu @ esAD nd @A P imges @ geAF (iii) P imge @ geD PHIIAD g ndD rr polriztionD ltrEpine mode @IFST m pixel spingAD geoodedD singleElookD shown in pigures TFV@A nd TFW@AF he P imge omes t IFST m pixel spingF o (t with the dydi deomE position imposed y the qudEtreeD we slightly resmpled the dt to otin the HFS a P GR m resolutionF hown smpling from IFST to P m is expeted to hve minor impt on the lssi(tion mpD euse the resmpling rtio is quite lose to unityF pive lsses re onsideredX urn @redAD wter @lueAD low vegettion @greenAD snd @yellowA nd ontiners @purpleAF e present the (nl lssi(tion mps in pigures TFIH nd TFII nd the orresponding lssi(tion uries on the test smples in le TFPF o the est of our knowledgeD none of the previously deE veloped pplitionEspei( methods llows diret integrtion of multiEresolutionD multiEfrequenyD nd multiEsensor optilEe dt without mjor preE or postE proessingF he results otined y the proposed multisensor hierrhil methodD see pigures TFIH@dA nd TFII@dAD led to detiled lssi(tion with remrkle level of lssi(tion mp regulrityF he min soure of mislssi(tion is the onE tiner reD where ontiners re prtly lssi(ed s urnF his is onsistent with the ft tht no texture fetures re used s input to the onsidered hierrhil wpF sn le TFP we ompre numerilly the results otined with the proposed hierrhil method when onsidering either only léidesD or oth e @gu or PA nd optil imgesF e oserve n improvement relted to the omintion of the two e imgesD in prtiulr in the urn res for whih the joint use of gu nd P quisition represents signi(nt soure of disrimintive informtionF wore spei(llyD we hve oserved tht the optil imge hs relevnt e'et in the snd nd vegettion disrimintionD nd the e quisitions re jointly very helpful to detet the urn reF his on(rms the potentil of exploiting the synE ergy of the dt provided y the gu nd P missions with respet to eh other nd to the imgery olleted y speEorne optil r mersF pigure TFWX eond site used for experimentsX @A léides @ gxi distriution eirus hAD @A gu @ esAD nd @A P imges @ geAF wter urn vegettion re soil ontiners over ll le TFPX glssi(tion uries of resultsF ixperiments were onduted on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF sn the proposed methodD multiEsensor nd multiEresolution fusion is sed on exE pliit sttistil modelingF st omines mrginl sttistil model of the onsidered input optil nd e imgesD through hierrhil wrkov rndom (eld modeling sed on qudEtrees in sdeD leding to sttistil supervised lssi(tion pE prohF e hve developed novel multiEsoure wwEsed hierrhil wrkov rndom (eld model tht tkes into ount oth e nd optil informtion nd leds to improved roustness of the lssi(erF hen pplied to severl hllenging highEresolution dt sets ssoited with urn nd semiEurn test sitesD the proE posed method gives high overll lssi(tion ury with smll omputtion time @ few minutesAF 7.1 Markov mesh random eld es disussed in ghpter R proilisti usl imge models hve een thoroughly studied sine the erly WH9s through hierrhil wps on qudEtreesF hese models rely on uslity onept ptured y the ftoriztion of the prior distriution in terms of usl trnsition proilitiesF sn prtieD this struture tends to generte 4loky4 e'ets in the (nl lssi(tion mpF everl tehniques hve een introE dued in etion RFQFP tht ould mitigte suh undesired e'etsF roweverD in these methodsD the prtil dvntges of the tree struture @iFeFD uslityA re then prE tilly @eFgFD ghrdin nd érezD IWWWA or ompletely lost @eFgFD uto et lFD IWWTD euse of the sptil intertions introdued in the respetive modelsAF sn this hpE terD the qudEtree pproh is extended nd novel ontextul multiEsle tehnique is proposed in order to lssify multiEresolution remote sensing dt tht inorE porte sptil ontextul informtion nd mitigte possile loky rtifts while keeping the uslity of the hierrhil modelF rereD the fous is on the fusion of multiEresolution nd sptilEontextul informtion for the supervised lssi(tion of singleEdte imgeryF por this purposeD let us mention nother importnt lss of rndom (eldsF wrkov mesh rndom (elds @wwpsAD or usl wrkov nE dom pields tht re lso known s nilterl wps @wpsA were (rst relled in eend et lFD IWTSD fesgD IWUPD ikrdD IWVHF es indited in ghpter QD imE ges re usully modeled on (nite retngulr lttie with eh site s ∈ S eing ssoited with one or more rndom vrilesF pigure UFIX egulr retngulr lttie X of size m × n o uild usl proess on this lttieD n order over the set of sites SD iFeFD wellEde(ned hrteriztion of the 4pst4 of site s s set of nodes preeding s @see etion QFSA is impliitly requiredF es shown in UFID X i,j is rndom (a) Second order MMRF (b) Third order MMRF pigure UFPX gusl neighorhoodF vrile ssoited with site s i,j loted t the oordintes (i, j) ∈ [1, n] × [1, m]F X = X 11 , X 12 , . . . , X mn represents the orresponding rndom (eldF he proility tht the rndom vrile X i,j tkes on the vlue x i,j is denoted y p(X i,j = x i,j ) D for ese of nottion this proility is revited s p(x i,j ) nd the glol proility is denoted y p(X = x) = p(X 11 = x 11 , X 12 = x 12 , . . . , X mn = x mn ) nd revited s p(x)F sn this ontextD X is sid to e n wwp ifX p(x i,j |pa(s i,j )) = p(x i,j |∂(s i,j )) @UFIA whereD pa(s i,j ) = {X a,b |a < i and b < j} is onsidered to e the pst of X i,j nd ∂(s i,j ) is suset of pa(s i,j ) tht presents n unilterl neighorhood of s i,j husD X is wwp with the neighorhood size of 2 if @see pigure UFPAX p(x i,j |pa(s i,j )) = p(x i,j |x i-1,j , x i,j-1 ) @UFPA es shown previously in etion QFSD if this property is ssumedD then the joint proility is equl to the produt of the proilities of eh pixel in the lttieD eh proility eing onditionl to the smll suset of the pst of the orrespondE ing siteF pormllyX

p(x) = m i=1 n j=1 p(x i,j |x i-1,j , x i,j-1 ) @UFQA
where the required oundry djustment is ful(lled y ssuming the sites outside of the (nite lttie to e zeroF his ftoriztion is wht mkes this model ttrtiveF eend et lFD IWTS proved tht wwp is lso wp while the inverse does not neessry holdsD iFeFD wwps form usl sulss of wpsF sn this hpterD novel lssi(tion method is proposed y integrting n exE pliit hierrhil grphEsed lssi(erD whih uses qudEtree struture to model multiEsle intertionsD nd n wwp to del with pixel wise ontextul inforE mtion in the sme sleF he hoies of qudEtree nd n wwp llow tking ene(t from their good nlytil properties @espeilly uslityA nd onsequently pply nonEitertive lgorithms tht fuse multiEresolution nd sptilEontextul inE formtion for lssi(tion purposesF 7.2 Methodology 7.2.1 Combined Structure sn this disserttionD novel hyrid struture tht omines sptil grid using usl wwp nd hierrhil wp vi qudEtree is employed to irumvents the loky rtifts of qudEtreeEsed methods nd inorporte sptil ontextul informtion in eh sleF he strting point is to give n order on the set of ll nodes SF por eh sle of the qudEtreeD usl wwp is integrted into the hierrhil strutureF eordinglyD node s t eh level of the qudEtree exept the root depends on one prent @t the level just oveA nd three neighors @t the sme levelAF por eh pixel in the root levelD there is no prent node nd only the neighors reminsF he shpes of the neighorhoods of the pixels in the top nd left orders of eh lttie t eh sle of the pyrmid re oviously dpted to the imge ordersF he resulting dependene grph is shown in pigure UFQF 7.2.2 MPM inference eginD when the uslity property holdsD nonEitertive lssi(tion lgorithms n e ppliedF sn prtiulrD reursive proedure on qudEtree is fesile for the ww riterionD in whih the im is to mximize the posterior mrginl t eh site sX ∀s ∈ S, x s = arg max xs∈Λ p(x s |y), @UFRA whih produes the on(gurtion tht mximizes the posteriori mrginl p(x s |y) over the (nite set Λ of themti lssesF (a) Graphical model for the hierarchical Markov Mesh (b) hierarchical structure pigure UFQX ryrid struture tht omines sptil grid using usl wwp nd hierrhil wp vi qudEtreeF o resolve reursively the optimiztion prolem in @UFRAD one ould use the sme onditionl independene ssumptions developed in etion SFP for the multiE temporl modelD the posterior mrginl p(x s |y) of the lel of eh node s ould e expressed s funtion of the posterior mrginl p(x s -|y) of the prent node s -in the qudEtree nd the posterior mrginls p(x s|y) where s is eh one of the sites in the set ∂(s) of the three usl neighors of sF pormllyX

p(x s |y) = x s -,∂(s) p(x s , x s -, x ∂(s) |y d(s) ) xs p(x s , x s -, x ∂(s) |y d(s) ) p(x s -|y) s∈∂(s) p(x s|y) @UFSA
where old fonts denote the mrginl posteriors of interest to the wwD s usulF Proof of equation ( 7.5)X

p(x s |y) = x s - p(x s |x s -, y)p(x s -|y) = x s -   ∂(s) p(x s |x s -, x ∂(s) , y)p(x ∂(s) |y)   p(x s -|y) = x s -   ∂(s) p(x s , x s -, x ∂(s) |y d(s) ) xs p(x s , x s -, x ∂(s) |y d(s) ) p(x ∂(s) |y)   p(x s -|y) = x s -   ∂(s) p(x s , x s -, x ∂(s) |y d(s) ) xs p(x s , x s -, x ∂(s) |y d(s) ) s∈∂(s) p(x s|y)   p(x s -|y) = x s -,∂(s) p(x s , x s -, x ∂(s) |y d(s) ) xs p(x s , x s -, x ∂(s) |y d(s) ) p(x s -|y) s∈∂(s) p(x s|y)
he equlities ross rows I nd P nd ross rows Q nd R derive from two ondiE tionl independene ssumptionsX A1. the distriution of the lels (x s , x s -, x ∂(s) ) given ll the oservtions y n e restrited to the distriution onditioned only to the desendnts of site sY A2. the lels s of the usl neighors ∂(s) re independent when onditioned to the dt yF hese ssumptions re nlogous to the onditionl independene ssumptions tht re ommonly epted when deling with @hierrhil or singleEsleA wpEsed imge nlysisD nd hve een mentioned in the previous hptersF hey re used within the proposed method for nlytil onvenieneF he formultion in @UFSA llows lulting reursively the posterior mrginl p(x s |y) t eh node s while the proilities p(x s , x s -, x ∂(s) |y d(s) ) re produedF husD this simpli(es to the determintion of the other proilities sed onX The likelihood term p(y s |x s ) is estimated using a nite Gaussian mixture as in Section 5.

p(x s , x s -, x ∂(s) |y d(s) ) = p(x s |x s -)p(x s -) p(x s )   s∈∂(s) p(x s |x s)p(x s) p(x s )   p(

Experimental results

sn this setionD we disuss preliminry results of the experimentl vlidtion of the developed ontextul hierrhil lssi(er sed on wwpD using pnhromti nd multiEspetrl léides imges quired over ortEuErine @ritiAF he (nest resolution of the multiEresolution pyrmid @level HA ws set equl to the (nest resolution of the input pnhromti imges @iFeF HFS mAF goEregistered multiEspetrl imges @t P mA were integrted t level P of the pyrmidF o (ll level ID wvelet deomposition of the pnhromti imge ws usedF pive lnd over lsses hve een onsideredX urn @redAD wter @lueAD vegettion @greenAD soil @yellowA nd ontiners @purpleAF sn the present workD mnully nnotted @provided y n expertA nonEoverlpping trining nd test sets were seleted in homogeneous resF ptilly disjoint trining nd test res were usedF es shown in pigures UFSD UFR nd le UFID the resulting lssi(tion mp shows tht the proposed hierrhil method leds to urte resultsD espeilly s ompred to the originl hierrhil lssi(tion tehnique sed on the ww riterion in vferté et lFD PHHH whih yields to 4loky4 lssi(tion @see pigures UFS@A nd UFR @AAF hese loky rtifts re mitigted y inorporting sptil ontextul informtionF he proposed method ws lso ompred to n extended version of the method in vferté et lFD PHHH in whih the loky rtifts were redued y employing prior updte tehnique in the topEdown step of the hierrhil lgorithm s desried in oisin et lFD PHIRF gompring to this methodD the proposed tehnique otins higher uries espeilly over the urn re inluded in the imged sene s shown in pigures UFRD UFS nd le UFIF he proposed method is imed t performing hierrhil lssi(tion using input multiEresolution imgeryF st omines usl hierrhil wp model usE ing qudEtree nd wrkov mesh to preserve ontextul informtion t eh sle y pplying nonEitertive lssi(tion lgorithm using the ww riterionF ixperimentl results with r stellite imgery suggest tht the method llows to e'etively inorportes sptil informtion in the hierrhil lssi(tion proE ess nd provides higher uries thn previous enhmrk tehniquesF roweverD gonsidering the oveD it is noted tht wwps nd their lttie models hve the following weknessX wwps re ornerEdependent nd onsequently the proposed hierrhil wp model sed on them re not neessrily symmetri @there exists n irregulrity issue with this wwp sed method due to the ft tht eh pixel depend on nonEsymmetri neighorhoodD s highlighted with lk rrows in pigE ures UFRD UFSAF o irumvent this drwk severl tehniques hve een introdued reently in the litertureF udrilterl wrkov ndom pield @wpA ws introE dued in zlighi et lFD PHHWD in whihD the nonEregulrity prolem is voided y enforing ll four wwp with respet to di'erent orners of the lttie into new (eld de(nitionY howeverD the model ws nonEsymmetriF o overome these limiE ttions from oth mthemtil nd prtil points of viewD ouse( et lFD PHIQ estlished new rndom (eldD symmetriD ornerEindependentD nd isotropi imE ge model whih inorportes the dependeny of pixel on ll its neighors using ymmetri wrkov wesh ndom pield @wwpAF his n e rehed y snE ning the lttie sed on spei( sheme s detiled in ouse( et lFD PHIQF he integrtion of wwp with n hierrhil wp ould e n interesting diretion of further reserhF

(a) (b) (c) (d)
pigure UFRX lssi(tion mps of optil@pléidesA imge @A using the originl method proposed in vferté et lFD PHHH @AD the proposed method @A nd the method in oisin et lFD PHIR @dAF (a) (b)

(c) (d)
pigure UFSX lssi(tion mps of optil@pléidesA imge @A using the originl method proposed in vferté et lFD PHHH @AD the proposed method @A nd method in oisin et lFD toint multiEdte nd multiEresolution fusion of optil dt ws ddressed on sis of expliit sttistil modelingF he method omined joint model of the input imges through hierrhil wp modelingD leding to sttistil supervised lssi(tion pprohF e novel wwEsed hierrhil wrkov rndom (eld model ws developed y onsidering multiEtemporl informtion ndD thusD supporting the lssi(tion of multiple imges tken over the sme re t di'erent times nd di'erent sptil resolutionsD prolem tht hs een very srely ddressed so fr in the dt fusion litertureF he results otined y the proposed method were nlyzed through experiments with multiEtemporl léides nd qeoiye dt setsF he experimentl results showed tht the method ws le to provide urte lssi(tion mps on oth dt setsF sndeedD the tehnique ws demonstrted to e dvntgeous in terms of the lssi(tion ury on the test setD the sptil regulrity of the lssi(tion mpsD the minimiztion of sptil rtiftsD nd the trdeo' with respet to omputtion time s ompred to previous enhmrk methods tht model multiEresolution or multiEtemporl or sptilEontextul informtionF hese results suggest the e'etiveness of the lgorithm in fusing oth multiEtemporl nd multiEresolution informtion for supervised lssi(tion purposes nd on(rm tht wp models represent powerful fusion tools in remote sensingF he methodologil rtionle of this multiEtemporl nlysis method hs een extended to the se of multiEsensor imge lssi(tionF he proposed method ddressed the prolem of joint supervised lssi(tion of multiEsensor imges inluding e nd optil omponents quired t multiple sptil resolutionsF e novel pprohD sed on multiple qudEtrees in sdeD to multiEsensor nd multiE resolution fusion ws developedF his pproh formlizes supervised fyesin lssi(er within this multiple qudEtree topology tht omines lssEonditionl sttistil model for pixelEwise informtion nd hierrhil wp for multiEsensor nd multiEresolution ontextul informtionF woreoverD fous on the spei( se of the fusion of multiEfrequeny e dt olleted y the gywyEkywed @ ndA nd eheeEP @g ndA sensors together with multiEspetrl léides dt ws relevnt to investigte the synergy mong the multiEfrequeny e nd multiEresolution informtion provided y these sensorsF purthermoreD nother usl tehnique hs een developed y omining usl hierrhil wp model using qudEtree nd wrkov mesh to preserve ontextul informtion t eh sleD lssify multiEresolution remote sensing dtD inorporte sptil ontextul informtionD nd mitigte possile loky rtifts while keeping the uslity of the hierrhil model nd pplying nonEitertive lgorithm using the ww riterionF ixperimentl results with r stellite imgery showed tht the method llows to e'etively inorporte sptil informtion in the hierrhil lssi(tion proess nd provides higher uries thn previous enhmrk tehniquesF he omputtionl dvntges of the hierrhil wpsD for whih ext reE ursive formultions of the ww deision rule re fesile with no need for timeE expensive wetropolis or qis smpling proeduresD hve een on(rmed y the experimentl results of the proposed methodsF ell the tehniques developed in this thesis re sed on wp models on spei( topology tht omprises multiple hierrhil qudEtreesD eh ssoited with n quisition dteD sensorD or omE intion of sensorsF his topology is )exile enough to ntively inorporte imges relted to distint sptil resolutionsD quisition timesD nd sensorsF st n lso ommodte the use of further multiEsle feturesF velet trnsforms re used to (ll in those levels of eh qudEtree tht re not ntively ssoited with input remote sensing dtF he seletion of the wvelet opertor mong lrge fmily of possile trnsforms is not ritil euse most trnsforms led to lssi(tion results with similr uriesF xeverthelessD huehies wvelets of order IH yields higher uries thn the other onsidered trnsformsF Perspectives Automation of the selection of the wavelet operator es future extension of the proposed methodsD the utomtion of the seleE tion of the wvelet opertor usingD for exmpleD ditionry of multiple trnsE forms woser et lFD PHII ould e inorported in the developed modelsF imE ilrlyD the pseudoElikelihood method ws used to optimize the smoothing pE rmeter of the otts sptil omponent of the proposed wp modelF elterE nte prmeter estimtion lgorithms sedD for exmpleD on menEsqure error erpio nd woserD PHHTD stohsti grdient ounesD IWVVD or wonteEgrlo tehE niques sáñez nd imóD PHHQ ould e integrted in the proposed method to dE dress the optimiztion of this prmeter nd of the prmeters involved in the trnsition proilities ross multiple slesD timesD or sensorsF he ury of the proposed method ws found sensitive to these prmetersF roweverD the experiE mentl results suggested thtD sed on the mening of these prmeters in reltion to temporl nd interEsle orreltionsD it is not di0ult for userGopertor to identify rnges on their vlues tht led to meningful models of the trnsition proilities nd yield urte resultsF SMMRF and quad-tree wrkov mesh rndom (elds nd their lttie models exhiit wellEknown wekE nessD iFeFD they re 4ornerEdependent4F gonsequentlyD the proposed integrtion of hierrhil wp nd sptil wrkov mesh model does not neessrily exhiit n nisotropi ehvior nd n e 'eted y the use of nonEregulr @nonEsymmetriA neighorhoodF o mitigte this drwk severl tehniques hve een reently introdued in the litertureF udrilterl wps were introdued in zlighi et lFD PHHWD in whih the nonEregulrity prolem is voided y using four wrkov meshes relted to the di'erent orners of the lttie nd enforing them into unique (eld de(nitionY howeverD the model is still nonEsymmetriF o overome these limittions from oth mthemtil nd prtil points of viewD ouse( et lFD PHIQ estlished new rndom (eldX symmetriD ornerE independentD nd isotropi model tht inorportes the dependeny of pixel on ll its neighors using symmetri wrkov mesh rndom (eldF he integrtion of this pproh with the proposed hierrhil tehniques ould e n interesting diretion of further reserhF Semantic classication sn this thesisD the numer of lsses for the proposed methods hs een kept (xed for ll levels of the hierrhil strutureF yn one hndD this is ommon prtie in supervised imge lssi(tionF yn the other hndD espeilly when r dt re onsideredD di'erent types of lnd over lsses my e ppreited t di'erent resolutionsF hereforeD further extension of this work would e to de(ne di'erent sets of lsses t distint levels of the pyrmid nd introdue hierrhil link etween these lsses ording to their semnti meningsF e pixelElevel lssi(E tion my not exeed the spetrl signture or ksttering fetures resulting from the iophysil nd geophysil properties of eh individul pixelD ut the neighorhood of the pixel rings sustntil informtion tht n e used to reE onstrut lndspe units nd funtionl resF henD vertil strtegy ould e ppliedX eh pixel ould e ssigned to n elementry lssD orresponding to the relted ground mterilD nd eh group of pixels onstitutes lsses s mixture of these mteril lssesF hereforeD pixel my elong to di'erent lsses ording to the sptil resolution of the imge in whih it elongsD nd semnti reltionE ship etween lsses my e de(ned nd would ritilly involve the vilility of multiEresolution ground truth dtF he extension of the proposed hierrhil wp model to this seEspei( typology of soft lssi(tionD for exmple sed on fuzzy resoningD will e further hllenging generliztion of the work desried in this thesisF v9espe des on(gurtions Ω = Λ |S| est l9ensemle de toutes les on(gurtions posE silesF X = {x s } s∈S , x s ∈ Λ et Y = {y s } s∈S sont onsidérés omme des proessus léE toiresF hns e ontexteD nous étudions le prolème d9inférene de l meilleure on(gurtion x de X dns ΩF v formultion yésienne de e prolème d9inférene onsiste à minimiser le oût d9erreur de segmenttion selon un ritère hoisi X x = arg min gel en remplçnt dns @fFIA l fontion du oût suivnte X

C(x, x ) = 1 -δ(x, x ) = 1 - R n=0 δ(x n , x n ) @fFQA
@dns le dre hiérrhiqueA où R est le nomre de niveux de l pyrmide et δ est le symole de uronekerF gette fontion de oût est égle à 1 si une erreur de lssi(tion pprît dns un niveu quelonque de l pyrmideF r onséqueneD ette fontion pénlise les erreurs indépendmment de l9éhelle à lquelle elles se produisentD e qui n9est ps très intéressnt pour des méthodes de lssi(tion hiérrhiques et ne ompte ps le nomre totl d9erreursF ve seond ritère très utilisé dns l littérture est elui des wodes wrginles osteriori @ww en nglisA dont l fontion du oût est dé(nie prX

C(x, x ) = s∈S 1 -δ(x s , x s ) @fFRA
A = p(x s |x s -, x s = , y d(s) ) = p(x s , x s -, x s = |y d(s) ) s p(x s , x s -, x s = |y d(s) )
@fFUA sl est à noter que les équtions dns @fFTA impliquent deux hypothèses d9indépendne onditionnelleX

• pour le pssge de l ligne I à l ligne P dns @fFTA X étnt donné les étiquettes x s -et x s = D l9étiquette x s ne dépend ps de tout l9ensemle d9oservtions yD elle dépend seulement des oservtions tthées à s et à ses desendnts e qui implique que X p(x s |x s -, x s = , y) = p(x s |x s -, x s = , y d(s) )

• pour le pssge de l ligne Q à l ligne R dns @fFTA X les étiquettes x s -et x s = sont indépendntesD étnt donné l9ensemle d9oservtion yD e qui implique X p(x s -, x s = |y) = p(x s -|y).p(x s = |y) sl reste à luler les proilités jointes p(x s , x s -, x s = |y d(s) ) évoquées dns @fFUA qui peuvent être déterminées en estimnt les proilités introduites dns l fE toristion suivnte X

p(x s , x s -, x s = |y d(s) ) = @fFVA p(x s |x s -, x s = ). p(x s -|x s = ).p(x s = ) p(xs)
.p(x s |y d(s) )

inlunt X

• des proilités priori @p(x s = ) et p(x s )AD

• des proilités interEéhelle etGou multiEsoure @p(x s |x s -) et p(x s |x s -, x s = )AD

• des proilités posteriori prtielles p(x s |y d(s) )F r l suiteD à hque nouvelle donnée t @nouvelle imge à une dte di'érente ou quise pr un pteur di'érentAD pour estimer l proilité mrginle posteriE ori p(x s |y)D il su0t de luler les proilités évoqueés dns @fFVAF e et e'etD l9pprohe proposée se déroule en trois psses réursivesD desendnte @IAD senE dnte @IA et desendnte @PA Passe descendante (1) : estimation des probabilités a priori e(n de luler les proilités priori dns l pyrmide P @à une nouvelle dte t dns le dre multiEtemporel ou ve un nouveu pteur c dns le dre multiE pteurAD nous proédons à une psse desendnteD en ommençnt pr estimer les proilités priori à l résolution l plus grossièreF our e fireD nous utilisons l rte de lssi(tion otenue en utilisnt l struture hiérrhique en sde jusqu9à l pyrmide P -1 en onsidérnt un modèle mrkovien sptil qui onduit à une meilleure estimtion de l9 prioriF einsiD en utilisnt le théorème de rmmersleyE gli'ordD nous pouvons dé(nir un priori lol pour hque pixel s à l résolution l plus grossière omme étnt un modèle de otts dérit iEdessous X Passe descendante (2) : Probabilité a posteriori et optimisation gonnissnt les proilités jointes lulées dns l prtie préédenteD nous proéE dons à une psse desendnte dns lquelle les proilités mrginles posteriori p(x s |y) sont estimées en utilisnt @fFTAD puis mximisées pr un lgorithme de wetropolis modi(é uto et lFD IWWTF hns les méthodes proposéesD l fusion multiErésolutionD multiEtemporelle ou multiEpteur est fondée sur une modélistion sttistique expliite u trvers d9un modèle hiérrhique de hmps de wrkovF v nouveuté priniple de l9pprohe est l9utilistion en sde de plusieurs qudErresD hun étnt ssoié à une nouvelle imge disponileD en vue de rtériser les orréltions temporelles ou sptiles ssoiées à des imges distintesF vorsqu9elle est ppliquée à des imges à hute résolutionD l méthode proposée donne une préision glole de l lssi(tion de vleur élevée ve un temps de lul risonnle grâe à l struture hiérrhique utiliséeF Bibliography eend et lFD IWTS eendD uFD rrleyD F tFD nd unlD vF @IWTSAF glssi(tion of inry rndom ptternsF IEEE Transactions on Information TheoryD II@RAXSQV! SRRF @gited on pges IIV nd IPHFA evijily et lFD PHHV evijilyD eFD il ueD sFD nd wngoudD wF @PHHVAF pusion of remote sensing imges using ontourlet trnsformF Innovations and Advanced Techniques in Systems, Computing Sciences and Software EngineeringD pges PIQ! PIVF @gited on pge QHFA ermin nd predD PHHR erminD F nd predD wF @PHHRAF yntheti perture rdrF Opt. Photon. NewsD IS@IIAXPV!QQF @gited on pge PQFA espert et lFD PHHU espertD pFD fhEgudrD wFD hirnD tFD gntoneD eFD nd rolezD pF @PHHUAF imeEvrying segmenttion for mpping of lnd over hngesF sn Proceeding of ESA Symposium, MontreuxF @gited on pge QSFA ezimiEdjdi nd fnnourD IWWI ezimiEdjdiD wF nd fnnourD F @IWWIAF woEdimensionl dptive lok ulmn (ltering of e imgeryF IEEE Transactions on Geoscience and Remote SensingD PW@SAXURP!USQF @gited on pge QQFA SensingD SH@IHAXQUHU!QUITF @gited on pge PVFA sáñez nd imóD PHHQ sáñezD wF nd imóD eF @PHHQAF rmeter estimtion in wrkov rndom (eld imge modeling with imperfet oservtionsF omprtive studyF Pattern recognition lettersD PR@IRAXPQUU!PQVWF @gited on pges WP nd IQIFA sngld nd werierD PHHU sngldD tF nd werierD qF @PHHUAF e new sttistil similrity mesure for hnge detetion in multitemporl e imges nd its extension to multisle hnge nlysisF IEEE Transactions on Geoscience and Remote SensingD RS@SAXIRQP!IRRSF @gited on pge QSFA sql et lFD PHIQ sqlD wFD qhfoorD eFD nd iddiquiD eF @PHIQAF tellite imge resolution enhnement using dulEtree omplex wvelet trnsform nd nonlol mensF IEEE Geoscience and Remote Sensing LettersD IH@QAXRSI!RSSF @gited on pge QHFA srving et lFD IWWU srvingD FD pieguthD FD nd illskyD eF @IWWUAF en overlpping tree pproh to multisle stohsti modeling nd estimtionF IEEE Transactions on Image ProcessingD T@IIAXISIU!ISPWF @gited on pge TUFA sshikwD PHHQ sshikwD rF @PHHQAF ixt optimiztion for wrkov rndom (elds with onvex priorsF IEEE Transactions on Pattern Analysis and Machine Intel-ligenceD PS@IHAXIQQQ!IQQTF @gited on pge RVFA tre nd nniD IWWR treD F nd nniD xF @IWWRAF higitl hnge detetion of forest onversion of dry tropil indin forest regionF International Journal of Remote SensingD IS@IQAXPSRQ!PSSPF @gited on pge QRFA tohnson nd usishkeD IWWV tohnsonD F nd usishkeD iF @IWWVAF ghnge veE tor nlysisX tehnique for the multispetrl monitoring of lnd over nd ondiE tionF International Journal of Remote SensingD IW@QAXRII!RPTF @gited on pge QIFA tolion nd osenfeldD PHIP tolionD tFEwF nd osenfeldD eF @PHIPAF A pyramid framework for early vision: multiresolutional computer visionD volume PSIF pringer iene 8 fusiness wediF @gited on pge THFA tordnD PHHQ tordnD wF @PHHQAF An introduction to probabilistic graphical modelsF Image registration for remote sensingF gmridge niversity ressF @gited on pge QPFA vi et lFD IWWS viD rFD wnjunthD fFD nd witrD F @IWWSAF wultisensor imge fusion using the wvelet trnsformF Graphical Models and Image ProcessingD SU@QAXPQS!PRSF @gited on pge PVFA vi et lFD PHII viD rFEgFD rongD FD uD FEFD nd pnD FEF @PHIIAF yn the empirilEsttistil modeling of e imges with generlized qmm distriuE tionF IEEE Journal of Selected Topics in Signal ProcessingD S@QAXQVT!QWUF @gited on pge IHPFA viD PHHW viD F @PHHWAF Markov random eld modeling in image analysisF pringer iene 8 fusiness wediF @gited on pges RID RT nd UVFA villesnd et lFD PHIR villesndD FD uieferD FD nd ghipmnD tF @PHIRAF Remote sensing and image interpretationF tohn iley 8 onsF @gited on pges IH nd IQFA viu et lFD PHIP viuD FEqFD hezertD tFD werierD qFD nd nD F @PHIPAF hynmi evidentil resoning for hnge detetion in remote sensing imgesF IEEE Transactions on Geoscience and Remote SensingD SH@SAXIWSS!IWTUF @gited on pge QPFA vuettgen et lFD IWWR vuettgenD wFD urlD FD nd illskyD eF @IWWRAF i0ient multisle regulriztion with pplitions to the omputtion of optil )owF IEEE Transactions on Image ProcessingD Q@IAXRI!TRF @gited on pges SI nd TUFA wdnin et lFD PHIR wdninD wFD o0ninD eFD nd rjinD eF @PHIRAF ghnge detetion through four tehniques using multiEtemporl vndst themti mpper dtX e se study on plvrjn reD ssfhnD srnF Journal of Environmental InformaticsD PQ@PAXSV!TTF @gited on pge QRFA wes et lFD IWWU wesD pFD gollignonD eFD ndermeulenD hFD wrhlD qFD nd uetensD F @IWWUAF wultimodlity imge registrtion y mximiztion of mutul informtionF IEEE Transactions on Medical ImagingD IT@PAXIVU!IWVF @gited on pge QPFA wlilD IWVH wlilD F @IWVHAF ghnge vetor nlysisX n pproh for deteting forest hnges with vndstF sn LARS SymposiaD pge QVSF @gited on pge QIFA wlltD PHHV wlltD F @PHHVAF A wavelet tour of signal processingF edemi pressD Q editionF @gited on pges UR nd VUFA wnndhr et lFD PHHW wnndhrD FD ydehD sF yFD nd enevD F @PHHWAF smE proving the ury of lnd use nd lnd over lssi(tion of vndst dt using postElssi(tion enhnementF Remote SensingD I@QAXQQH!QRRF @gited on pge QTFA wrhesi et lFD PHIH wrhesiD FD fovoloD pFD nd fruzzoneD vF @PHIHAF e ontextE sensitive tehnique roust to registrtion noise for hnge detetion in r mulE tispetrl imgesF IEEE Transactions on Image ProcessingD IW@UAXIVUU!IVVWF @gited on pge QRFA wrroquin et lFD IWVU wrroquinD tFD witterD FD nd oggioD F @IWVUAF roE ilisti solution of illEposed prolems in omputtionl visionF Journal of the American Statistical AssociationD VP@QWUAXUT!VWF @gited on pge TRFA wrtinis nd weleD PHIH wrtinisD F nd weleD eF @PHIHAF e hierrhil sptioE temporl wrkov model for improved )ood mpping using multiEtemporl End e dtF Remote SensingD P@WAXPPRH!PPSVF @gited on pge QSFA ws nd ploresD PHHV wsD tF nd ploresD tF @PHHVAF he pplition of rti(il neurl networks to the nlysis of remotely sensed dtF International Journal of Remote SensingD PW@QAXTIU!TTQF @gited on pge QHFA welgni et lFD PHHP welgniD pFD woserD qFD nd erpioD F @PHHPAF nsuperE vised hngeEdetetion methods for remoteEsensing imgesF Optical EngineeringD RI@IPAXQPVV!QPWUF @gited on pge QRFA welgni nd erpioD PHHQ welgniD pF nd erpioD F @PHHQAF e wrkov rndom (eld pproh to sptioEtemporl ontextul imge lssi(tionF IEEE Transactions on Geoscience and Remote SensingD RI@IIAXPRUV!PRVUF @gited on pges QID QTD WHD WPD WRD WS nd WTFA wetropolis et lFD IWSQ wetropolisD xFD osenluthD eFD osenluthD wFD ellerD eFD nd ellerD iF @IWSQAF iqution of stte lultions y fst omputing mhinesF

The Journal of Chemical PhysicsD PI@TAXIHVU!IHWPF @gited on pge RTFA willer et lFD IWWS willerD hFD uminskyD iFD nd nD F @IWWSAF xeurl network lssi(tion of remoteEsensing dtF Computers & GeosciencesD PI@QAXQUU!QVTF @gited on pge QTFA wilneD IWVV wilneD eF @IWVVAF ghnge diretion nlysis using vndst imgeryX review of methodologyF sn Geoscience and Remote Sensing Symposium, 1988D

volume ID pges SRI!SRRF @gited on pge QPFA wolinier nd usteD PHHU wolinierD wF nd usteD F @PHHUAF gomprison nd evlution of polrimetri hnge detetion tehniques in eril e dtF sn IGARSSD pges PQVT!PQVWF @gited on pge QSFA wontopoli et lFD PHIQ wontopoliD wFD giminiD hFD vmnteD wFD rerzogD wFD qrfD rFD nd wrznoD pF @PHIQAF wirowve rdiometri remote sensing of volni sh louds from speX wodel nd dt nlysisF IEEE Transactions on Geoscience and Remote SensingD SI@WAXRTUV!RTWIF @gited on pge QPFA woser et lFD PHII woserD qFD engitiD iFD nd erpioD F @PHIIAF wultisle unE supervised hnge detetion on optil imges y wrkov rndom (elds nd wveletsF IEEE Geoscience and Remote Sensing LettersD V@RAXUPS!UPWF @gited on pges QR nd IQHFA wuñozEwrí et lFD PHIH wuñozEwríD tFD fovoloD pFD qómezEghovD vFD fruzzoneD vFD nd gmpEllsD qF @PHIHAF emisupervised oneElss support vetor mhines for lssi(tion of remote sensing dtF IEEE Transactions on Geoscience and Remote SensingD RV@VAXQIVV!QIWUF @gited on pge QHFA xeelmniD PHHQ xeelmniD F @PHHQAF snverse prolems in imge proessingF Rice University, TexasF @gited on pge RHFA xelsonD IWVQ xelsonD F @IWVQAF heteting forest nopy hnge due to inset tivity using vndst wF Photogrammetric Engineering and Remote SensingD RW@WAXIQHQ!IQIRF @gited on pge QPFA xenini et lFD PHHU xeniniD pFD qrzelliD eFD frontiD FD nd elproneD vF @PHHUAF emote sensing imge fusion using the urvelet trnsformF Information FusionD V@PAXIRQ!ISTF @gited on pge QHFA xguyen et lFD PHII xguyenD xFD xsrdiD xFD nd rnD F @PHIIAF oust multiEsensor lssi(tion vi joint sprse representtionF sn International Conference on Information FusionD pges I!VF @gited on pge QHFA risiD IWVV risiD qF @IWVVAF Statistical eld theoryF eddisonEesley puF @gited on pge RHFA érezD IWWQ érezD F @IWWQAF Champs markoviens et analyse multirésolution de l'image: application à l'analyse du mouvementF hh thesisF @gited on pges QH nd SPFA érez et lFD PHHH érezD FD ghrdinD eFD nd vfertéD tFEwF @PHHHAF xoniterE tive mnipultion of disrete energyEsed models for imge nlysisF Pattern RecognitionD QQ@RAXSUQ!SVTF @gited on pges SID TH nd VIFA erez nd reitzD IWWT erezD F nd reitzD pF @IWWTAF estrition of wrkov rndom (eld on grph nd multiresolution sttistil imge modelingF IEEE Transactions on Information TheoryD RP@IAXIVH!IWHF @gited on pge SHFA etitjen et lFD PHII etitjenD pFD uetterlinD eFD nd qnçrskiD F @PHIIAF e glol verging method for dynmi time wrpingD with pplitions to lusterE ingF Pattern RecognitionD RR@QAXTUV!TWQF @gited on pges QI nd QPFA ikrdD IWVH ikrdD hF @IWVHAF nilterl mrkov (eldsF Advances in Applied ProbabilityD pges TSS!TUIF @gited on pge IIVFA iellD PHHQ iellD qF @PHHQAF edptive wvelets nd their pplitions to imge fusion nd ompressionF @gited on pge QHFA ohl nd vn qenderenD PHIR ohlD gF nd vn qenderenD tF @PHIRAF emote sensE ing imge fusionX n updte in the ontext of digitl erthF International Journal of Digital EarthD U@PAXISV!IUPF @gited on pge PVFA ohlD IWWV ohlD gFnd n qenderenD tF @IWWVAF eview rtile multisensor imE ge fusion in remote sensingX oneptsD methods nd pplitionsF International Journal of Remote SensingD IW@SAXVPQ!VSRF @gited on pge QHFA rendesD PHIS rendesD tF @PHISAF New statistical modeling of multi-sensor images with application to change detectionF hh thesisD niversité risEudD prneF @gited on pge QHFA iner et lFD IWVW inerD vFD veeD gFD tungD fFD nd ilponD tF @IWVWAF rww lustering for onneted word reognitionF sn International Conference on Acoustics, Speech, and Signal Processing, 1989.D pges RHS!RHVF @gited on pge SQFA zlighi et lFD PHHW zlighiD FD uehtrnvzD xFD nd xosrtiniD eF @PHHWAF gomputtion of imge sptil entropy using qudrilterl wrkov rndom (eldF IEEE Transactions on Image ProcessingD IV@IPAXPTPW!PTQWF @gited on pges SID IPS nd IQIFA ihrdsD PHIQ ihrdsD tF @PHIQAF Remote sensing digital image analysisF pringerD S editionF @gited on pge QQFA oerts et lFD PHHV oertsD tFD n erdtD tFD nd ehmedD pF @PHHVAF essessE ment of imge fusion proedures using entropyD imge qulityD nd multispetrl lssi(tionF Journal of Applied Remote SensingD P@IAF @gited on pge QHFA okingerD IWWT okingerD yF @IWWTAF ixelElevel fusion of imge sequenes using wvelet frmesF sn Proceedings of the 16th Leeds applied shape research workshop, Leeds University PressF giteseerF @gited on pge QHFA os et lFD PHIS osD FD pernndesD hFD xogueirD tF fFD nd immerD gF @PHISAF eutomti hnge detetion in multitemporl End End e imges using qrmEhmidt proessF sn Geoscience and Remote Sensing SymposiumD pges PUWU!PVHHF @gited on pge QRFA osenfeld et lFD IWUT osenfeldD eFD rummelD FD nd ukerD F @IWUTAF ene leling y relxtion opertionsF IEEE Transactions on Systems, Man and Cy-berneticsD @TAXRPH!RQQF @gited on pge RVFA vitzky nd qolyD IWTR vitzkyD eF nd qolyD wF @IWTRAF moothing nd difE ferentition of dt y simpli(ed lest squres proeduresF Analytical chemistryD QT@VAXITPU!ITQWF @gited on pge QQFA howengerdtD PHHT howengerdtD F @PHHTAF Remote sensing: models and methods for image processingF edemi pressF @gited on pge QQFA erpio et lFD PHIP erpioD FD hellepineD FD foniD qFD woserD qFD engitiD iFD nd udriD F @PHIPAF snformtion extrtion from remote sensing imges for )ood monitoring nd dmge evlutionF Proceedings of the IEEED IHH@IHAXPWRT! PWUHF @gited on pge SFA erpio nd woserD PHHT erpioD F nd woserD qF @PHHTAF eight prmeter opE timiztion y the ro!ushyp lgorithm in mrf models for supervised imge lsE si(tionF IEEE Transactions on Geoscience and Remote SensingD RR@IPAXQTWS! QUHSF @gited on pges WQ nd IQIFA err et lFD PHHQ errD FD onsD FD nd uriD hF @PHHQAF ostElssi(tion hnge detetion with dt from di'erent sensorsX some ury onsidertionsF International Journal of Remote SensingD PR@ITAXQQII!QQRHF @gited on pge QTFA hh et lFD PHHV hhD FD ounnD xFD nd uingD F @PHHVAF en e0ient pnE shrpening method vi omined dptive ge pproh nd ontourletsF IEEE Transactions on Geoscience and Remote SensingD RT@SAXIQPQ!IQQSF @gited on pge QHFA inghD IWVW inghD eF @IWVWAF higitl hnge detetion tehniques using remotelyE sensed dtF International Journal of Remote SensingD IH@TAXWVW!IHHQF @gited on pges IRHW!IRIPF @gited on pge QHFA winD IWUV winD F @IWUVAF fyesin lssi(tion in timeEvrying environE mentF @gited on pge QTFA zirnyi nd hdydehD PHIR zirnyiD F nd hdydehD wF @PHIRAF egmentE tion of remote sensing imges using similrityEmesureEsed fusionEmrf modelF IEEE Geoscience and Remote Sensing LettersD II@WAXISRR!ISRVF @gited on pge QIFA okol et lFD IWWW okolD FD vöfmnD FD nd irkkiläD eF @IWWWAF eltive liE rtion of multitemporl vndst dt for forest over hnge detetionF Remote Sensing of EnvironmentD TV@IAXI!IIF @gited on pge QPFA ouzi et lFD PHHW ouziD FD heshmpsD eFD nd otherD qF @PHHWAF hse of trget sttering for wetlnd hrteriztion using polrimetri gEnd eF IEEE Transactions on Geoscience and Remote SensingD RU@WAXQPRI!QPTIF @gited on pge QSFA delhovenD PHII delhovenD F @PHIIAF imesttsX e softwre tool for the retrievl of temporl ptterns from glol stellite rhivesF IEEE Journal of Selected Topics in Applied Earth Observations and Remote SensingD R@PAXQIH!QIUF @gited on pge QQFA n de oorde et lFD PHHU n de oordeD FD he qenstD FD nd gntersD pF @PHHUAF smproving pixelEsed r lndEover lssi(tions of urn res with postElssi(tion tehniquesF Photogrammetric Engineering and Remote SensingD UQ@WAXIHIUF @gited on pge QTFA iteriD IWTU iteriD eF @IWTUAF irror ounds for onvolutionl odes nd n symptotilly optimum deoding lgorithmF IEEE Transactions on Information TheoryD IQ@PAXPTH!PTWF @gited on pge SRFA ivone et lFD PHIS ivoneD qFD elproneD vFD ghnussotD tFD hll wFD wFD qFD eFD viirdiD qFD estinoD FD nd ldD vF @PHISAF e ritil omprison mong pnshrpening lgorithmsF IEEE Transactions on Geoscience and Remote Sens-ingD SQ@SAXPSTS!PSVTF @gited on pge PVFA oisinD PHIP oisinD eF @PHIPAF Classication supervisée d'images d'observation de la Terre à haute résolution par utilisation de méthodes markoviennesF hh thesisD niversity of xie ophi entipolisD prneF @gited on pges QD QHD TUD UH nd UIFA oisin et lFD PHIP oisinD eFD urylovD FD woserD qFD erpioD FD nd eruiD tF @PHIPAF wultihnnel hierrhil imge lssi(tion using multivrite opulsF sn IS&T/SPIE Electronic ImagingF snterntionl oiety for yptis nd hotonE isF @gited on pges QHD VS nd WIFA oisin et lFD PHIR oisinD eFD urylovD FD woserD qFD erpioD FD nd eruiD tF @PHIRAF upervised glssi(tion of wultiEsensor nd wultiEresolution emote ensing smges with rierrhil gopulEsed epprohF IEEE Transactions on Geoscience and Remote SensingD SP@TAXQQRT!QQSVF @gited on pges SID THD UHD UID VID IHID IHTD IHUD IHVD IHWD IPSD IPTD IPU nd IPVFA ldD IWWW ldD vF @IWWWAF ome terms of referene in dt fusionF IEEE Transactions on geoscience and remote sensingD QU@QAXIIWH!IIWQF @gited on pge PVFA ltz et lFD IWWH ltzD iFD vlinsD tFD et lF @IWWHAF Multisensor data fusionD volume TVSF erteh house fostonF @gited on pges PV nd QHFA ng nd vingD PHIR ngD hF nd vingD F @PHIRAF smproving ves mpping y integrting modis nd ylopes li produts using optiml interpoltionF IEEE Journal of Selected Topics in Applied Earth Observations and Remote SensingD U@PAXRRS!RSUF @gited on pge PVFA ske nd vn der vindenD PHHV skeD fF nd vn der vindenD F @PHHVAF glsE sifying multilevel imgery from e nd optil sensors y deision fusionF IEEE Transactions on Geoscience and Remote SensingD RT@SAXIRSU!IRTTF @gited on pge QHFA eer et lFD PHIP eerD tFD etitjenD pFD nd qnrskiD F @PHIPAF owrds efE (ient stellite imge time series nlysisX gomintion of dynmi time wrping nd qusiE)t zonesF sn IEEE Geoscience and Remote Sensing SymposiumD pges RQVU!RQWHF @gited on pge QPFA hittkerD PHHW hittkerD tF @PHHWAF Graphical models in applied multivariate statisticsF iley uF @gited on pges RWD SI nd SQFA illskyD PHHP illskyD eF @PHHPAF wultiresolution wrkov models for signl nd imge proessingF Proceedings of the IEEED WH@VAXIQWT!IRSVF @gited on pges QHD SI nd TRFA ng et lFD PHHU ngD fFD viD FD nd unD pF @PHHUAF smge fusion using nonsuE smpled ontourlet trnsformF sn IEEE International Conference on Image and GraphicsD pges UIW!UPRF @gited on pge QHFA ounesD IWVV ounesD vF @IWVVAF istimtion nd nneling for qisin (eldsF sn Annales de l'IHP Probabilités et statistiquesD volume PRD pges PTW!PWRF @gited on pge IQIFA ouse( nd uehtrnvzD PHII ouse(D F nd uehtrnvzD xF @PHIIAF qenertE ing symmetri usl wrkov rndom (eldsF Electronics lettersD RU@PPAXIF @gited on pge SIFA ouse( et lFD PHIQ ouse(D FD uehtrnvzD xFD nd goD F @PHIQAF gomputE tionlly trtle stohsti imge modeling sed on symmetri mrkov mesh rndom (eldsF IEEE Transactions on Image ProcessingD PP@TAXPIWP!PPHTF @gited on pges IPS nd IQIFA hng nd rongD PHHS hngD F nd rongD qF @PHHSAF en sr nd wvelet inE tegrted pproh to improve pnEshrpening visul qulity of nturl olour suyxy nd uikfird imgesF Information FusionD T@QAXPPS!PQRF @gited on pge QHFA

  IU IFTFI ghrteristis of remote sensing imgesF F F F F F F F F F F F IU IFTFP ssive vsF etive ensing F F F F F F F F F F F F F F F F F F F F IV IFTFQ ssive sensors operting in the optil spetrum F F F F F F F IV IFTFR etive sensors operting in the mirowve spetrum F F F F F PH IFU emote sensing pplitions F F F F F F F F F F F F F F F F F F F F F F F PS 2 Data fusion in Remote sensing 27 PFI ht fusion in emote sensing F F F F F F F F F F F F F F F F F F F F F PV PFP wultiEsensor nd multiEresolution pprohes F F F F F F F F F F F F F PV PFPFI wultiEresolution pprohes F F F F F F F F F F F F F F F F F F F PV PFPFP wultiEsensor pprohes F F F F F F F F F F F F F F F F F F F F F QH PFQ wultiEtemporl nlysis pprohes F F F F F F F F F F F F F F F F F F F QI PFQFI wultiEtemporl fusion t feture level F F F F F F F F F F F F F QP PFQFP wultiEtemporl fusion t deision level F F F F F F F F F F F F F QS3 Markovian image models 39 QFI snverse prolems nd ontextul informtion F F F F F F F F F F F F F F RH QFIFI he ssing model F

  SI QFSFI puntionl hrteriztion of the uslity F F F F F F F F F F SP QFSFP qrphil hrteriztion of the uslity F F F F F F F F F F F SP QFSFQ gusl yptimiztion tehniques on wrkov hin F F F F F F F SQ iv Contents 4 Hierarchical Markov Models on quad-trees 59RFI fsi model struture F F F F F F F F F F F F F F F F F F F F F F F F F F TH RFP istimtion nd inferene lgorithm on qudEtree F F F F F F F F F F F TP RFPFI rolem sttement F F F F F F F F F F F F F F F F F F F F F F F F TP RFPFP ww inferene F F F F F F F F F F F F F F F F F F F F F F F F F FTS RFQ floky rtifts nd further extensions of the hierrhil model F F TU RFQFI floky rtifts F F F F F F F F F F F F F F F F F F F F F F F F F F TU RFQFP yther tehniques using trees F F F F F F F F F F F F F F F F F F F TW 5 Multi-temporal hierarchical Markov model on quad-tree 73

  VT SFRFI ht sets nd experimentl setup F F F F F F F F F F F F F F F F VT SFRFP ixperimentl esults nd omprisons F F F F F F F F F F F F F WH 6 Multi-sensor hierarchical Markov model on quad-tree 99 TFI wultiEsensor hp modeling F F F F F F F F F F F F F F F F F F F F F F F IHH TFP he (rst proposed multiEsensor hierrhil model F F F F F F F F F F IHP TFPFI methodology F

Contents 3 . 1

 31 Inverse problems and contextual information . . . . . . . . . 40 3.1.1 The Ising model . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.1.2 The Potts model . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2 General characteristics of Markovian image models . . . . . 41 3.3 Inference and Bayesian estimation . . . . . . . . . . . . . . . 44 3.3.1 Bayesian estimators . . . . . . . . . . . . . . . . . . . . . . . 45 3.3.2 Optimization approaches . . . . . . . . . . . . . . . . . . . . 46 3.4 Graphical representation of Markovian models . . . . . . . . 48 3.4.1 Dependency graph . . . . . . . . . . . . . . . . . . . . . . . . 49 3.4.2 Graphical interpretation . . . . . . . . . . . . . . . . . . . . . 49 3.5 Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.5.1 Functional characterization of the causality . . . . . . . . . . 52 3.5.2 Graphical characterization of the causality . . . . . . . . . . . 52 3.5.3 Causal Optimization techniques on Markov chain . . . . . . . 53

  systemsX @A (rstEorderY @A seondEorderF Denition 3.2.3 (Markov random eld) Let Xbe a collection of random variables X indexed by the elements of a set of sites S. X is a Markov random eld (MRF) with respect to a neighborhood system N on S if:

  gonditionl independene is n importnt onept s it n e used to deompose omplex proility distriutions into produt of ftorsD eh onsisting of the suset of orresponding rndom vrilesF sn the se of regulr lttieD the grph tkes prtiulrly simple form lled Markov blanket s shown in pigure QFRX node is onditionlly independent of ll other nodes when onditioned only on the neighoring nodesF husD the wrkov lnket of node simply onsists of the set of ll neighoring nodesF pigure QFRX pirst order wrkov lnketF 3.4.1 Dependency graph vet us denote X wrkovin proessF o re)et the intertion struture indued y the energy funtion U (X = x)D one ould de(ne dependeny grph s followsX Denition 3.4.2 (Dependency graph) With the same notation as in Section 3.2 the dependency graph associated with the energy decomposition U (x) = c∈C V c (x) is the graph G = (S, E) such as:

  ee vuritzenD IWWTD vuritzen et lFD IWWHD hittkerD PHHW for more disussion out dependeny grphs nd their semntisF 3.4.2 Graphical interpretation sn this setionD we exmine the struturl trnsformtions generted on indepenE dene grphs y the following si opertionsF Freezing he independenes n e red o' from the grph y freezing suset of vrilesF es shown in pigure QFSD n lterntive wy to view the onditionl independene etween to susets of verties A nd B onditioned to C @A ⊥ B | CAD is to imgine removing ll nodes in set C from the grph together with ny links tht onnet to those nodesF e then sk if there exists pth tht onnets ny node in e to ny node in fF sf there re no suh pthD then the onditionl independene property holdsF pigure QFSX qrphil onsequene of the onditionl independeneF Summing and maximizing umming nd mximizing t similr wy s freezing on the grphF sndeedD st hs een estlished in erez nd reitzD IWWT thtD given wrkovin proessD X ssoited to grph G(S, E)D the mrginl p(x A ) for some suset A ⊂ S hs n independene grph G D in whih two sites re neighors if they re neighors in GD or if they elong to the neighorhood of sme onneted omponent A = S \ AF his results from the summtion of p(x) = p(x A , x A ) with respet to x A whih provides the relted mrginl distriution pieguthD PHIHF es onsequene of this resultD from grphil viewpointD dependeny grph of the mrginl distriution p(x A ) is derived from the dependeny grph ssoited with the joint distriution p(x) y removing ll nodes a ∈ A nd then y mking ll neighors of a mutully neighor s shown in pigureQFTF pigure QFTX qrphil onsequene of summingGmximizing out the vriles in eF sing the grphil representtionD probabilistic causal models hve een lredy thoroughly studied vuettgen et lFD IWWRD érez et lFD PHHHD vferté et lFD PHHHD illskyD PHHPF

  y suessively onsidering mrginls of vetors x pa(N ) , x pa(N -1) , • • • , x 1 )D ghrdinD PHHH estlished funE tionl ses where usl representtion turns out to e t lest s lol s the originl nonEusl one @iFeFD ∂ i = V i ∩ pa(i)AF3.5.1 Functional characterization of the causalityes shown in the etion QFRFPD the onstrution of the dependeny grph ssoited with mrginl distriution p(x 1 , • • • , x i ) from the dependeny grph ssoited with the joint distriution p(x 1 , • • • , x N ) involves the retion of edges etween nodes neighoring the set of removed sitesF roweverD in ertin ses depending on the expression of the ftors under onernD simpli(tions might our in the mrginl p(x 1 , • • • , x i )D leding to simpler independene grph érezD IWWQF Proposition 3.5.1 (Functional characterization) We denote by X a Markovian process with the joint distribution p(x) = p(x 1 ) N i=2

  sition QFSFP with respet to n ordered set re triangulated @or hordlAD iFeFD they ontin no yles of length 4 without hordF ixmples of tringulted grphs re shown in pigure QFUF pigure QFUX ixmples of tringulted grphsF 3.5.3 Causal Optimization techniques on Markov chain vet us denote {(X t , Y t )} t∈[1,n] hidden wrkov hin tht respets the grph in pigure QFV pigure QFVX ridden wrkov hinF where f (., .) denote the trnsition proilities nd the e(., .) re emission proE ilitiesF pormlly X

  x p(X = x, Y = y) pirst let us onsider the following propositionX Proposition 3.5.3 ∀a, b and ∀ u, v bounded above functions: if u(a) 0 and v(a, b) 0 then: max a,b [u(a)v(a, b)] = max a u(a) max b v(a, b) et time tD let us onsider ll the pths in the sequene of sttes tht reognize the oservtions until time tD (Y 1 = y 1 , .

Contents 4 . 1

 41 Basic model structure he ojetive of this hpter is to desrie multiEresolution lssi(tion tehniques sed on hierrhil wrkovin modelingF hese methods hve two requirements for omplishing this tskX (i) the method should e le to hndle the dt quired t di'erent resolutions (ii) the method should provide struture tht n esily hndle the intertions etween di'erent imges in the input dt setF rllel multiEgrid @or pyrmidlA shemes re one of the possile pprohes stisfying requirement (i)F he pyrmid struture is type of signl representtion in whih imges re orgnized ording to their resolutions tolion nd osenfeldD PHIP @see pigure RFIAD iFeFD pyrmid P is stk of imE ges I n for whih the sle n ∈ [0, R] nd R is the height of the pyrmidF en element of this pyrmid is lled node nd my orrespond to pixel or group of pixels in the imge dominF pigure RFIX smges re orgnized ording to their resolutions in pyrmid struE tureF o hndle requirement (ii)D we de(ne for eh node of the pyrmid set of links to other nodes to model sleEtoEsle intertionsF he theE ory of multiEsle signls hs een widely studiedD nd their representtions nturlly led to models of signls on treesF emong othersD dydi trees @eFgFD fsseville et lFD IWWPD fsseville et lFD IWWPA nd qudEtrees @eFgFD érez et lFD PHHHD vferté et lFD PHHHD ghrdinD PHHHD oisin et lFD PHIRA hve een proposed s ttrtive ndidtes for modeling these sleEtoEsle intertions (a) (b)

  rtifts using the method proposed in vferté et lFD PHHH using the ww riterion t vrying hight of the qudEtreeF 4.3.2 Other techniques using trees 4.3.2.1 Kato et al.'s method uto et lFD IWWT onsider more omplex grph @lled Qh pyrmidl wpA in whih the originl qudEtree is omined with sptil lttie neighorhood t eh level s shown in pigure RFSF pigure RFSX xeighorhood system in uto et lFD IWWTF

Contents 5 . 1

 51 Multi-temporal structure . . . . . . . . . . . . . . . . . . . . . 74 5.2 Multitemporal MPM inference . . . . . . . . . . . . . . . . . 76 5.2.1 Time t = 0: single-time MPM. . . . . . . . . . . . . . . . . . 78 5.2.2 Time t = 1: rst top-down pass. . . . . . . . . . . . . . . . . 79 5.2.3 Time t = 1: bottom-up pass. . . . . . . . . . . . . . . . . . 79 5.2.4 Time t = 1: second top-down pass. . . . . . . . . . . . . . . 80 5.2.5 Both times: combination with MMD. . . . . . . . . . . . . . 80 5.3 Pixelwise class-conditional PDFs and transition Probabilities 82 5.3.1 Pixelwise class-conditional PDFs . . . . . . . . . . . . . . . . 82 5.3.2 Transition probabilities . . . . . . . . . . . . . . . . . . . . . 84 5.4 Experimental results . . . . . . . . . . . . . . . . . . . . . . . 86 5.4.1 Data sets and experimental setup . . . . . . . . . . . . . . . . 86 5.4.2 Experimental Results and comparisons . . . . . . . . . . . . . 90

  (a) Training set of urban area class (b) Urban area modeling pigure SFQX ixmple of hp modeling using the iw lgorithm on pnhromti léides imge quired on ortEuErineD ritiD gxiD distriution eirus hF Algorithm 5.3.1 (Stochastic Expectation Maximization (SEM) algorithm)

5. 4

 4 Experimental results 5.4.1 Data sets and experimental setup sn this setionD we disuss the results of the experimentl vlidtion of the developed multiEtemporl nd multiEresolution lssi(er on two dtsets @exmples re shown in pigures SFR nd SFSAX • e threeEdte series of pnhromti nd multispetrl léides imges E quired over ortEuErine @ritiA in PHIID PHIPD nd PHIQF • wo pnEshrpened qeoiye quisitions quired over ortEuErine @ritiA in PHHW nd PHIHF (a) 2011 (b) 2012 (c) 2013 pigure SFRX ixmples of imges time series quired over ortEuErine @ritiA y léides stellite gxiD distriution eirus hF (a) 2009 (b) 2010

  ury of the proposed method on the test set of the léides dt set s funtion of the prmeters @A ϕ nd @A θF (a) Detail on the Pléiades image (b) ϕ = 0.45 and θ = 0.7 (c) ϕ = 0.2 and θ = 0.85 (d) ϕ = 0.45 and θ = 0.85

  (a) Pléiades image (2013) (b) Ground Truth (training+test sets) (c) [Laferté et al., 2000] (with MPM) (d) [Laferté et al., 2000] (with MAP) (e) [Melgani and Serpico, 2003] (f) The proposed method (g) K-NN+MRF (h) K-means pigure SFIIX glssi(tion mps otined from léides dt setD @ gxi distriE ution eirus hAF (a) GeoEye image (2010) (b) Ground Truth (training+test sets) (c) [Laferté et al., 2000] (with MPM) (d) [Laferté et al., 2000] (with MAP) (e) [Melgani and Serpico, 2003] (f) The proposed method (g) K-NN+MRF (h) K-means pigure SFIPX glssi(tion mps otined from qeoiye dt set @ qeoiyeAF

  p(y|θmn) = p(y|νmn, σmn, κmn)

  opt ): p(x s |y d(s) ) ∝ p(y s |x s )p(x s ) ⇑ Bottom-up pass, (s ∈ S n opt , n = 1, . . . , R): p(x s |y d(s) ) ∝ p(y s |x s )p(x s ) u∈s + xu p(x u |y d(u) ) p(x u ) p(x u |x s ) Initialization, (s ∈ S R opt ): p(x s |y) = p(x s |y d(s) )

3. 1 ⇓

 1 Top-down pass II, (s ∈ S n , n = 0, 1, . . . , R -1):p(x s |y) = x s -,x ∂(s) p(xs,x s -,x ∂(s) |y d(s) ) xs p(xs,x s -,x s = |y d(s) ) p(x s -|y) s∈∂(s)p(x s|y)

@

  gited on pge SRFA utoD IWWS utoD tF @IWWSAF Modelisations markoviennes multiresolutions en vision par ordinateur. Application a la segmentation d'images SPOTF hh thesisD niversity of xie ophi entipolisD prneF @gited on pges QH nd VHFA uto et lFD IWWT utoD FD ferthodD wFD nd eruiD tF @IWWTAF e hierrhil wrkov rndom (eld model nd multitemperture nneling for prllel imge lssi(tionF Graphical Models and Image ProcessingD SV@IAXIV!QUF @gited on pges QD TUD TWD IIV nd IQWFA uto nd eruiD PHIP utoD F nd eruiD tF @PHIPAF Markov random elds in image segmentationF xow uF @gited on pges RTD TR nd UVFA uersten et lFD PHHS uerstenD FD veeD tFD nd einsworthD F @PHHSAF e ompriE son of hnge detetion sttistis in yve imgesF ehnil reportD hsg houmentF @gited on pge QSFA uun et lFD IWVS uunD hFD whukD eFD trndD FD nd ghvelD F @IWVSAF edptive noise smoothing (lter for imges with signlEdependent noiseF IEEE Transactions on Pattern Analysis and Machine IntelligenceD @PAXITS!IUUF @gited on pge QQFA uurum et lFD PHII uurumD wFD vngD FD y9xeillD FD tosephD eFD tksonD FD nd goshD wF @PHIIAF e (rstEorder rditive trnsfer model for mirowve rdiomeE try of forest nopies t lEndF IEEE Transactions on Geoscience and Remote SensingD RW@WAXQITU!QIUWF @gited on pge QPFA vferté et lFD PHHH vfertéD tFEwFD érezD FD nd reitzD pF @PHHHAF hisrete wrkov imge modeling nd inferene on the qudtreeF IEEE Transactions on Image ProcessingD W@QAXQWH!RHRF @gited on pges QD QHD SID THD TRD TSD TUD TVD TWD UHD UVD UWD VHD VSD WHD WID WRD WSD WTD IHTD IHUD IPQD IPSD IPTD IPUD IPVD IQTD IQU nd IQWFA vmin nd trhlersD IWWR vminD iF nd trhlersD eF @IWWRAF ghngeEvetor nlysis in multitemporl speX tool to detet nd tegorize lndEover hnge proesses using high temporlEresolution stellite dtF Remote Sensing of Envi-ronmentD RV@PAXPQI!PRRF @gited on pge QRFA vndgreeD PHHS vndgreeD hF @PHHSAF Signal theory methods in multispectral remote sensingD volume PWF tohn iley 8 onsF @gited on pge IHFA vuritzenD IWWT vuritzenD F @IWWTAF Graphical modelsF yxford niversity ressF @gited on pges RW nd SIFA vuritzen et lFD IWWH vuritzenD FD hwidD eFD vrsenD fFD nd veimerD rF @IWWHAF sndependene properties of direted wrkov (eldsF NetworksD PH@SAXRWI!SHSF @gited on pges RW nd SIFA ve woigne et lFD PHII ve woigneD tFD xetnyhuD xFD nd istmnD F @PHIIAF

  pge QIFA ingh et lFD PHIR inghD FD utoD FD nd eruiD tF @PHIRAF e multilyer mrkoE vin model for hnge detetion in eril imge pirs with lrge time di'erenesF sn IEEE International Conference on Pattern Recognition (ICPR)D pges WPR!WPWF @gited on pge QIFA owellD IWWP owellD pF @IWWPAF wximum likelihood estimtion of sttionry univrite frtionlly integrted time series modelsF Journal of econometricsD SQ@IAXITS!IVVF @gited on pge QRFA torvik et lFD PHHW torvikD fFD torvikD qFD nd pjortoftD F @PHHWAF yn the omE intion of multisensor dt using metEqussin distriutionsF IEEE Transactions on Geoscience and Remote SensingD RU@UAXPQUP!PQUWF @gited on pges IHID IHTD IHUD IHV nd IHWFA troppin et lFD PHIS troppinD hFD ezrD FD gloD pFD epeD eFD smpertoreD FD foshettiD wFD ilvD tFD frivioD FD nd vnriD F @PHISAF emote sensing of urned reX e fuzzyEsed frmework for joint proessing of optil nd miE rowve dtF sn IEEE, Geoscience and Remote Sensing Symposium (IGARSS)D

  -resolution Analysis o enhne results nd to use ll ville dt t

	Chapter 2
	Data fusion in Remote sensing
	Contents
	2.1 Data fusion in Remote sensing . . . . . . . . . . . . . . . . . 28 their di'erent sptil resolution 2.2 Multi-sensor and multi-resolution approaches . . . . . . . . 28
	Multi-temporal Analysis y using the vlule sptioEtemporl informtion reE 2.2.1 Multi-resolution approaches . . . . . . . . . . . . . . . . . . . 28
	sulting from multiEtemporl imge quisitions in order to nlyze phenomen 2.2.2 Multi-sensor approaches . . . . . . . . . . . . . . . . . . . . . 30
	hppened on the irth surfe during given time periodF 2.3 Multi-temporal analysis approaches . . . . . . . . . . . . . . 31
	2.3.1 Multi-temporal fusion at feature level . . . . . . . . . . . . . 32 Change Detection o determine the hnges from imges tken t di'erent times on the sme reF 2.3.2 Multi-temporal fusion at decision level . . . . . . . . . . . . . 35

  arg max

	xt henD Forward nd Backward proilities hve een introdued vi the deompoE L t sition of @QFQPAX = x t-1 p(x t , x t-1 , y 1 , . . . , y t ) = x t-1 p(x t , y t |x t-1 )p(x t-1 , y 1 , . . . , y t-1 ) = x t-1 p(y t |x t )p(x t |x t-1 )F t-1 = e(y t |x t ) x t-1 f (x t , x t-1 )F t-1 L @QFQQA B

t = p(x t )p(y|x t ) @QFQPA = p(x t , y 1 , . . . , y t ) × p(y t+1 , . . . , y n |x t ) = F t × B t where F t nd B t re omputed reursively s followsX F t = p(x t , y 1 , . . . , y t ) t = p(y t+1 , . . . , y n |x t ) @QFQRA = x t+1 p(x t+1 , y t+1 , . . . , y n |x t ) = x t+1 p(x t+1 |x t )p(y t+1 , . . . , y n |x t+1 ) = x t+1 f (x t+1 , x t )p(y t+1 |x t+1 )p(y t+2 , . . . , y n |x t+1 ) = x t+1 f (x t+1 , x t )e(y t+1 , x t+1 )B t+1 Algorithm 3.5.2 (Baum-Welch algorithm) ⇒ Forward pass Initialization

  in wrroquin et lFD IWVUD foumnD IWWID ww well dpts the estimtor to the qudEtree topologyF sndeedD euse the tree is yliD the lE els re estimted reursively y ww through forwrdEkwrd lgorithm simiE lr to the lssil fum nd elh lgorithm for wrkov hins fum et lFD IWUHF sn the following of this thesis ww riterion is hosen to ddress inferene prolems on the qudEtree struturesF 4.2.2 MPM inference he strting point of the originl proedure introdued in vferté et lFD PHHH lies in the expression of the posterior mrginl p(x s |y) s funtion of the posterior mrginl t the prent node p(x s -|y)F pormlly it hs een shown tht under some independene onditionl ssumptions detiled in vferté et lFD PHHHXp(x s |y) = , x s -|y d(s) ) p(x s -|y).@RFWA here y d(s) is the vetor olleting the oservtions of ll the desendnts of site s ∈ S n , n = 0, 1, . . . , R -1F his yields topEdown reursion wheres the posterior mrginl is initilized t the root node s well s the proilities p(x s , x s s |x s ) re the pixelwise lssEonditionl hps of the imge dt t eh node of the qudEtreeF xote tht the produt over the hildren set is tully sent t the leves of the tree @iFeFD s ∈ S 0 A in whih the prtil posterior mrginls re omputed diretlyX p(x s |y s ) ∝ p(y s |x s ) p(x s ). Blocky artifacts and further extensions of the hierarchical model67 Algorithm 4.2.1 (MPM estimation on the quad-tree)This step provides the distributions p(x s |y d(s) ) and p(x s , x s -|y d(s) ) via (4.10) and (4.12) respectively.

	M P M s purthermoreD s shown x s -A preliminary top-down recursion is performed to compute the prior = arg max xs∈Λ p(x s |y), @RFVA p(x s , x s -|y d(s) ) t∈s + xt p(x t |y d(t) ) p(x t ) p(x t |x s ), @RFIPA where p(y Preliminary pass probabilities using (4.11).

xs p(x s -|y d(s) ) re mde villeF woreoverD these proilities n e omputed sX p(x s , x s -|y d(s) ) = p(x s |x s -)p(x s -) p(x s ) p(x s |y d(s) ), @RFIHA where nother onditionl independene ssumption hs een usedD p(x s |x s -) is the trnsition proility over sleD nd p(x s )D p(x s -) re the prior mrginls omputed using simple top down reursionX p(x s ) = x s - p(x s |x s -) p(x s -). @RFIIA e ottomEup pss llows to ompute the prtil posterior mrginls p(x s |y d(s) ) in RFIHF vferté et lFD PHHH proved tht these mrginls ould e expressed s reursion over sles @s ∈ S n , n = 1, 2, . . . , RAX p(x s |y d(s) ) ∝ p(y s |x s )p(x s ) @RFIQA he whole proedure is summrized in pigure RFQ nd elgorithm RFPFIF

(a) (b) pigure RFQX he reursive lgorithm on the qudEtree vi the ww riterion 4.3. ⇑ Bottom-up pass

  SFPX glssi(tion uries on the test set of the léides dtsetX lss uriesD overll uryD nd omputtion timeF ixperiments were onduted using one @ITHHxIHHHA imge t level HD one @VHHxSHHA imge t level I nd four @RHHxPSHA nds t level P on n sntel iU qudEore @PFRH qrzA VEqfEew TREit vinux systemF

		urban water vegetation bare soil containers over all computation time
	Proposed method	81.62	100	90.69	92.82	62.82	85,59	480 seconds
	method in [Laferté et al., 2000] using MPM criterion 77.45 88.62	72.59	86.02	57.02	76.34	160 seconds
	method in [Laferté et al., 2000] using MAP criterion 56.14	100	81.90	87.02	73.21	79.65	220 seconds
	method in [Melgani and Serpico, 2003]	80.63	100	86.33	87.61	69.61	84.83	≈ 1 hour
	K-NN + MRF	96.84 92.42	47.15	71.83	16.75	64.99	90 seconds
	K-means	12.37 98.63	59.18	91.66	29.42	58.25	20 seconds
		urban water vegetation bare soil containers over all computation time
	Proposed method	87.59	100	98.12	72.82	82.27	88,16	345 seconds
	method in [Laferté et al., 2000] using MPM criterion 77.45	100	88.34	66.22	67.87	79.97	160 seconds
	method in [Laferté et al., 2000] using MAP criterion 64.52	100	92.15	85.62	49.47	78.35	140 seconds
	method in [Melgani and Serpico, 2003]	80.63	100	89.79	70.54	74.29	83,05	≈ 1 hour
	K-NN + MRF	100	100	0	0	12.28	42.45	40 seconds
	K-means	88.97	100	88.14	45.6	36.96	71.93	15 seconds

le

  x s |y d(s) ) @UFTA sn @UFTAD the (rst ftor p(x s |x s -) orresponds to the hildEprent trnsition proE ilityY p(x s ) is the prior proilityY p(x s |x s) is the usl neighorhood trnsition proility in the sme sleY nd p(x s |y d(s) ) is the prtil posterior mrginl proE ilityF Proof of equation (7.6)Xp(x s , x s -, x ∂(s) |y d(s) ) = p(x s -, x ∂(s) |x s )p(x s |y d(s) ) = p(x s -|x s )p(x ∂(s) |x s )p(x s |y d(s) )= p(x s -|x s ) distriution of the lels x s -nd x ∂s re independent on the oservtions y d(s) of the desendnts of sD when onditioned to the lel x s of s iFeFD p(x s -, x ∂s |x s , y d(s) ) = p(x s -, x ∂s |x s ) @row IA A4. he distriution of the lels x s -nd x ∂s re independent when onditioned to x s iFeFD p(x s -, x ∂(s) |x s ) = p(x s -|x s )p(x ∂(s) |x s ) @from row I to row PA A5. he distriution of the lels x s -nd x s re independent when onditioned tox s D iFeFD p(x s -, x ∂(s) |x s ) = p(x s -|x s )p(x ∂(s) |x s ) @from row P to row QA hese results llow the reursive tehnique in elgorithm UFPFI to e formultedF st provides generliztion of the hierrhil wpEsed lssi(tion method proE posed in vferté et lFD PHHH nd of the multiEtemporl tehnique in ghpter Q tht llows sptilEontextul informtion to e inorported through the wwp omE ponentD in ddition to multiEresolution informtion through the qudEtree hierrhyF por trnsition proilities nd lssEonditionl hpsD the sme models nd estiE mtes s in ghpter RD whih re sed on the model desried in foumnD IWWID on (nite qussin mixturesD nd on iwD re extended here s wellF Denition of the inter-scale transition probability p(x s |x s -) using the formulation introduced in [Bouman, 1991], and extension of the same formulation to the inter-neighborhood transition probability p(x s |x s). Initialization of the prior p(x s ) at the root s ∈ S R using Potts model as discussed in Section 5.2.1. Top-down pass I, (s ∈ S n , n = 0, 1, . . . , R -1): (x s |x s -) p(x s -) Initialization, (s ∈ S 0 ): p(x s |y d(s) ) ∝ p(y s |x s )p(x s ) ⇑ Bottom-up pass, (s ∈ S n , n = 1, . . . , R): p(x s |y d(s) ) ∝ p(y s |x s )p(x s ) Initialization, (s ∈ S R ): p(x s |y) = p(x s |y d(s) )

	  p(x s |x s -)p(x s -) s∈∂(s) p(x s ) where the equlity ross rows I nd P nd ross rows P nd Q derives from the p(x s|x s )   p(x s |y d(s) ) =   s∈∂(s) p(x s |x s)p(x s) p(x s )   p(x s |y d(s) ) following ssumptionsX Initialization, (s ∈ S R ): p(x s ) = x s -A3. Algorithm 7.2.1 (Contextual hierarchical technique) u∈s + xu p(x u |y d(u) ) p(x u ) p(x u |x s )

he ⇓ p

  to the ttined very high sptil resolutions @up to few dozens of entimetersAD the short revisit timesD nd the joint vilility of dt with di'erent sptil resolutions nd overgesF sn this frmeworkD the present hFhF thesis foused on the development of novel multiEmodl dt fusion tehniques for the joint supervised lssi(tion of imges olleted on the sme geogrphil re t di'erent times y distint sensors t multiple sptil resolutions in di'erent wvelength rngesF everl hllenging methodologil prolems hve een ddressed to support the joint exploittion of multiple remote sensing dt modlities t one for the genertion of themti lssi(tion resultsF he mthemtil formliztions of the proposed tehniques re rooted in the re of hierrhil multiEsle nd usl rndom (eldsF hey im t tking ene(t from the omplementry informtion onveyed y remote sensing oservtions with diverse resolutionsD sensorsD frequeniesD nd quisition timesD t formulting their fusion in )exile nd nlytilly rigorous wyF

	Conclusion and perspectives
	Conclusion
	gontemporry stellite missions for irth oservtion through optil @pnhroE
	mti nd multiEspetrlA nd e sensors urrently o'er unpreedented
	oservtion pilitiesD thnks
		wter urn vegettion ontiners soil over ll omputtion time
	roposed method	IHH	WP	VW	VI	WR	WI	IRU seonds
	method in vferté et lFD PHHH	IHH	TP	UT	UP	WI	VH	IPH seonds
	method in oisin et lFD PHIR	IHH	UR	VQ	VT	WP	VU	ISR seonds
	le UFIX esults otined using the léides dtsetX lss uriesD overll uryD nd omputtion timeF
	PHIR @dAF

  'eter à hque pixel s l9étiquette optimle xs D en mximisnt réurE sivement l proilité mrginle posteriori p(x s |y)D el en exprimnt l proE ilité mrginle du pixel s à trvers l proilité mrginle p(x s -|y) du prent s -F wlgré ses performnes en temps de lul et en préisionD l9utilistion de et lgorithme onduit à des rtéfts de los vferté et lFD PHHHF ge phénoméne peut être expliqué pr le fit queD à une éhelle donnéeD deux pixels voisinsD peuE vent ne ps voir le même prentF hns e sD une limite pprît euoup plus filement que s9ils étient liés u même prent ou s9il existit d9utres liisons enE tre les pixels à des éhelles onséutivesF e(n d9éviter es rtéftsD nous pouvons utiliser une struture hiérrhique plus onnetée dns lquelle l liison usle sendntEdesendnt dns l même pyrmide n9est ps l9unique liison entre les niveux onséutifs @fF pr exemple foumnD IWWIAF hns e ontexteD une nouvelle formultion de l9lgorithme fondé sur le ritère ww est proposée en utilisnt l struture hiérrhique en sde dé(nie dns l prtie ??F v proilité mrginle posteriori p(x s |y) de hque pixel sD est exprimée non seulement à trvers l proilité mrginle posteriori p(x s -|y) du prent s -D mis ussi à trvers l proilité mrginle posteriori p(x s = |y) du prent à une soure ntérieure s = (n de rtériser les orréltions entre des imges onséutives de l struture en sde dé(nie et éviter insi les rtéfts de los X p(x s |y) = s -,s = p(x s |x s -, x s = , y).p(x s -, x s = |y) = s -,s = p(x s |x s -, x s = , y d(s) ).p(x s -, x s = |y)

	qui vise à =	A . p(x s -, x s = |y)
	s -,s =
	=	A . p(x s -|y).p(x s = |y)	@fFTA
	s -,s =
	où
	v9utilistion de ette fontion o're l9vntge de pénliser les erreurs en fontion de
	leur nomre et de l9éhelle à lquelle elles se produisentX pr exempleD une erreur à
	une éhelle grossière est plus fortement pénlisée qu9une erreur à une éhelle plus
	(neF v9estimteur yésien résultnt est dé(ni pour hque pixel s prX
	∀s ∈ S, xs = arg max xs∈Λ	p(x s |y)	@fFSA
	rmis les méthodes utilisées pour résoudre e prolème d9optimistionD on distingue
	l méthode fondée sur l struture qudErre employée pr vferté et lFD PHHH

  est le symole de uronekerD s ∼ s signi(e que s et s sont voisins pr rpport à un système de voisinge donné et β est un prmètre de lissgeD de signe positif pour fire déroitre l9énergie lorsque les étiquettes sont identiquesF uisD une psse desendnte est e'etuée pour luler les proilités priori à hque niveu du qudErre en utilisnt l formultion réursive suivnte X trnsition interEéhelle u sein de l9rre p(x s |x s -) peuvent être lulées indépendmment de telle sorte que x s pprtienne à l même lsse que son sendnt x s -F Passe ascendante (1) : estimation des probabilités jointes in se snt sur @fFVAD les proilités jointes p(x s , x s -, x s = |y d(s) ) sont déterminées pr X

	p(x s |x s , s ∼ s) ∝ exp -β s -p(x s |x s -) • p(x s -) s∼s δ (x s , s s ) ves proilités de p(x s) ves proilités posteriori prtielles néessitent l9estimtion d9un modèle sttisE @fFWA @fFIHA • p(x s, x s ) @fFIIA tique onditionnel pr lsse ominnt des informtions pixel pr pixelD à l même résolution pour le lul de p(y s |x s )F gel peut se fire en tritnt l modélistion des proilités mrginles des imges quises à hque résolutionD onditionnées yù δ(.) p(x s ) = à hque lsseF

• les proilités priori p(x s )D déjà estimées lors de l psse desendnte prééE denteD

• les proilités interEéhelle etGou interEpyrmide p(x s |x s -) et p(x s |x s -, x s = )D qui peuvent

être lulées en fvorisnt l9pprtenne des pixels onernés à une même lsseY • les proilités posteriori prtielles p(x s |y d(s) ) qui peuvent être lulées moyennnt une psse sendnte en utilisnt l formultion introduite pr vferté et lFD PHHHX p(x s |y d(s) ) ∝ p(y s |x s ) • p(x s ) • s∈s + s p(x s|y d(s) )

pigure PFIX qenerl dt fusion rhitetureF

Well-posedness of a problem stems from a denition given by[Hadamard, 1923].A problem, involving some mathematical model of physical phenomena, is Hadamard well-posed if a solution exists, is unique and its behavior changes continuously with the initial conditions ; otherwise, it is ill-posed.

A complete graph is a simple undirected graph in which every pair of distinct vertices is connected by a unique edge.

Dynamic programing refers to optimization methods that allows an exact inference technique[Bellman, 1956].

(a) (b) (c) (d)pigure TFIHX @A esult using only optil léides imgesD @A esult using oth léides nd gu quisitionsD @A esult using oth léides nd P quisitions @dA esult using ll sensorsF 114 Chapter 6. Multi-sensor hierarchical Markov model on quad-tree(a) (b) (c) (d)pigure TFIIX @A esult using only optil léides imgesD @A esult using oth léides nd gu quisitionsD @A esult using oth léides nd P quisitions @dA esult using ll sensorsF

Appendix A Scientic activity he following ppers hve een pulished or sumitted on works ontined in this disserttionX A.1 Peer-reviewed papers for international conferences

• sF redhliD qF woserD tF erui et F fF erpioD 4gontextul multiEsle imge lssi(tion on qudEtree4D siii snterntionl gonferene on smge roessE ing @sgsAD PHITD @soumisAF

• sF redhliD qF woserD tF eruiD nd F fF erpioD 4xew hierrhil joint lssi(tion method of eEoptil multiresolution remote sensing dt4D siiiGies iuropen ignl roessing gonferene @isgyAD xieD prneD eug PHISF

• sF redhliD qF woserD tF eruiD nd F fF erpioD xew sde model for hiE errhil joint lssi(tion of multisensor nd multiresolution remote sensE ing dtF siii snterntionl qeosiene nd emote ensing ymposium @sqeAD tul PHISD wilnD stlyF PHISF

• sF redhliD qF woserD tF eruiD nd F fF erpioD 4pusion of multitemporl nd multiresolution remote sensing dt nd pplition to nturl dissters4D in siii snterntionl qeosiene nd emote ensing ymposium @sqeAD uéeD gndD tuly PHIRF

• sF redhliD qF woserD tF eruiD nd F fF erpioD 4xew sde model for hierE rhil joint lssi(tion of multitemporlD multiresolution nd multisensor remote sensing dt4D in siii snterntionl gonferene on smge roessing @sgsAD risD prneD ytoer PHIRF A.2 Peer-reviewed journals

• sF redhliD qF woserD tF eruiD nd F fF erpioD 4e xew gsde wodel for the rierrhil toint glssi(tion of wultitemporl nd wultiresolution emote ensing ht4D siii rnstions on qeosiene nd emote ensing @qA @under revisionAF

• sF redhliD qF woserD tF eruiD 4xouvelle méthode en sde pour l lssiE (tion hiérrhique multiEtemporelle ou multiEpteur d9imges stellitires hute résolution4D evue prnçise de hotogrmmétrie et de élédétetion @pA @sumittedAF A.3 International Summer School

• vs wwi gryyv @wy PHISD rievyxsusD qiigiA inviE ronmentl eurityX emote ensing of xturl esoures

• qR wwi gryyv @tul PHISD eseD sevA ht pusion for isk wpping Appendix B

Résumé étoé

ves moyens mis en oeuvre pour surveiller l surfe de l erreD notmment les zones urinesD en s de tstrophes nturelles telles que les inondtions ou les tremlements de terreD et pour évluer l9impt de es événementsD jouent un rôle primordil du point de vue soiétlD éonomique et huminF he très nomreux usges @sienti(quesD ivils et militiresA sont fits de es donnéesF ves stellites ont eu une ontriution importnte et unique dns ertins domines dont l9évlution et l surveillne de l9environnement @déforesttionD pollutionD pollution lumineuseD urnistion et périurnistionD frgmenttion éologiqueD érosionD etFAF hns e dreD des méthodes de lssi(tion préises et e0es sont des outils prtiulièrement importnts pour ider à l9évlution rpide et (le des hngements u sol et des dommges provoquésF Étnt données l9énorme quntité et l vriété des données disponiles grâe ux missions stellitires de dernière génértion et de di'érents typesD l priniple di0ulté est de trouver un lssi(eur qui puisse prendre en ompte des données multiEndeD multiErésolutionD multiEdteD et éventuellement multiEpteurD tout en grdnt un temps de lul risonnleF gonsidérnt une série d9imges stellitiresD un modèle multiEsoure est proposé (n de fusionner les informtions sptilesD temporelles et multiErésolutionF e ette (nD deux pprohes yésiennes peuvent être doptées X un lssi(eur en sde otenu en supprimnt le ouplge entre les dimensions sptiles et temporellesD et en lssi(nt hque imge de l série sur l se d9elleEmême et des imges préédentes ou l9pprohe mutuelle qui lssi(e hque imge sur l se des imges préédentes et ultérieuresF v9pprohe en sde est doptée dns et thése (n de préserver l9ordre temporel des imges et d9méliorer l lssi(tion à hque nouvelle donnée disponileF v méthode proposée orde le prolème de l lssi(tion multiEtemporelle et multiErésolution dns lquelleD l fusion multiEdte et multiErésolution est fondée sur une modélistion sttistique expliite u trvers d9un modèle hiérrhique de hmps de wrkov @wpA fusionnnt l9informtion sptioEtemporelle et multiErésolutionF Classieur hiérarchique proposé v9ojetif de l lssi(tion est d9estimer un ensemle d9étiquettes hées X shnt un ensemle d9oservtions Y tthées à l9ensemle des noeuds @pixelsA SF ghque étiquette une vleur dns l9ensemle des étiquettesD Λ = {0, 1, ..., M -1}F