{"id":1937,"date":"2017-10-18T11:56:16","date_gmt":"2017-10-18T15:56:16","guid":{"rendered":"https:\/\/www.flagshipsoftware.com\/wordpress\/portal\/?page_id=196"},"modified":"2022-05-24T11:50:12","modified_gmt":"2022-05-24T15:50:12","slug":"duplicate-extraction","status":"publish","type":"page","link":"https:\/\/www.flagshipsoftware.com\/fsweb\/duplicate-extraction\/","title":{"rendered":"Duplicate Extraction"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\n\t\n\t\n\t\n\t\n<\/svg>\t\t<\/div>\n\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Priority Duplicate Extraction<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Duplicate extraction utility allows you to identify and selectively remove duplicate records from your database. If you merge 2 or more files, you can identify records that are common between them. \u00a0This is useful for removing suppression lists, also known as Kill Files.<\/p>

From the Address Management menu, select \u201cPriority Duplicate Extraction\u201d. Click \u201cProcess\u201d to preform the extraction.\u00a0<\/p>

By default, iAddress\u2122 checks for duplicate records based on address. You can customize your duplication by choosing some, all or no parts of the address. \u201cCity\u201d is not selected by default to take into account alternate city names (for example Elk Falls, BC\/Campbell River, BC or North York, ON\/Downsview, ON).<\/span><\/p>

\u00a0<\/span>You can also include other fields to identify duplicates. These are as follows:<\/span><\/p>

  • First Name<\/span><\/strong><\/li><\/ul>

    All records with the same First Name will be identified. \u00a0First Name will only be available if you have bound the first name field when creating your job.<\/span><\/p>

    • Last Name<\/span><\/strong><\/li><\/ul>

      Records with the same Last Name will be identified. \u00a0Last Name will only be available if you have bound the Last Name field when creating your job.<\/span><\/p>

      • Name Line (1, 2, 3)<\/span><\/strong><\/li><\/ul>

        Records containing the same information in the appropriate name line will be marked as duplicates.<\/span><\/p>

        • Partial Names<\/span><\/strong><\/li><\/ul>

          Records are identified if the names are similar. For example, Commercial Bank, Commercial Bank of Canada and Commercial would be considered duplicates as long as the addresses were the same.<\/span><\/p>

          • Ignore Rural Addresses<\/span><\/strong><\/li><\/ul>

            By selecting this option, iAddress\u2122 will not check rural addresses for duplicates.<\/span><\/p>

            • Blank and non-blank lines are not duplicate<\/span><\/strong><\/li><\/ul>

              If no data is contained in a field, iAddress\u2122 will consider this to be a unique entry and will not match it to non-blank entries.<\/span><\/p>

              • Records with all blank names are not part of duplicate processing<\/span><\/strong><\/li><\/ul>

                Any records where all the name fields are blank are left out of the process while checking for duplicates.
                <\/span><\/p>

                \u00a0<\/span>Percent match<\/span><\/strong><\/p>

                The percent value entered in this box allows the user to determine how similar two name fields must be before they are identified as duplicates. The higher the percent, the more exactly two name fields must match. To find exact matches, the percent should be set at 100. Lowering the percent value will allow for spelling mistakes.<\/span><\/p>

                \u00a0<\/span>Processing duplicates<\/span><\/strong><\/p>

                To check your database for duplicates, select the \u201cProcess\u201d button and the progress bar will be displayed above the buttons.\u00a0 All records identified as duplicates are flagged; common records are those identified in your suppression list (Kill File) that appear in any other databases merged into your job.<\/span><\/p>

                \u00a0When processing is complete, you can choose to view your duplicates or \u00a0common records or delete them.<\/span><\/p>

                \u00a0<\/span>Duplicate report<\/span><\/strong><\/p>

                Click Print Report to produce a detailed description of your duplicate extraction results.<\/span><\/p>

                \u00a0<\/span>To view duplicates<\/span><\/strong><\/p>

                Selecting the View\/Print \u00a0will open the View Database screen with the Duplicates selected. From this screen, you can print a list of duplicate addresses or selectively delete them. \u00a0 You may also flag or de-flag records as a duplicate.<\/span><\/p>

                \u00a0Deleting duplicates<\/span><\/strong><\/p>

                Duplicates may be automatically removed using Delete Duplicate Addresses.<\/span><\/p>

                Note:<\/span><\/strong><\/p>

                iAddress\u2122 will automatically retain the first occurrence of the addresses and delete all additional duplicates found.
                Information about duplicates can be exported using the Extra Control Fields option on the Export Database screen.<\/span><\/p>

                To run your job<\/span><\/strong><\/p>

                After selecting all your options, click \u201cProcess\u201d.
                To stop this process while it is running, click \u201cCancel\u201d.<\/span><\/p>

                Using a kill file<\/strong><\/span><\/p>

                Kill\/Do Not Mail files (suppression lists) are used to remove addresses from the main database(s). If you have merged more than 2 files, you will need to identify the kill file. Click the \u201cSelect Kill File\u201d button and choose the appropriate file. \u00a0After processing, all records matching the Kill file will be marked as common. \u00a0You can view, print and\/or export these records. \u00a0When you are ready to remove the Kill file and all matching records, click subtract.<\/span><\/p>

                Setting priorities for duplicates<\/strong><\/span><\/p>

                You can decide which of your duplicate identification criteria is more important by setting priorities.<\/span><\/p>

                \u00a0Deduping before correction<\/span><\/strong><\/p>

                If you process your duplicates before correcting your file, you should choose the Perform Precorrection option. This option corrects and standardizes your records to produce the best results.<\/span><\/p>

                \u00a0Once your Criteria and Options are selected, click Set Priority. The following screen will appear. You can determine which options are most important by moving them up and down the list. If you don\u2019t want to use a particular option, you can remove them. You can move items related to each option in the right hand box. You can also delete them, if they are not important to your duplicate extraction.<\/span><\/p>

                \u00a0Length<\/span><\/strong><\/p>

                This option selects the duplicate with the most number of characters. For example, if this option were selected for the First Name field, JONATHON would be considered a higher priority duplicate than J or JOHN. When you delete your duplicates, JONATHON would be retained while J and JOHN would be deleted.<\/span><\/p>

                File order<\/span><\/strong><\/p>

                If you have merged two or more files, you can determine which file is more important when determining duplicates. For example, if the third file you merge into your job is your main customer list, you can select this as the priority. If duplicates are found in your file, duplicates found in your main customer list will be considered a higher priority. When you delete your duplicates, the record from this file would be retained while the others will be deleted.<\/span><\/p>

                Language<\/span><\/strong><\/p>

                This option will prioritize duplicates by language. If English has been selected as the priority, French duplicates will be deleted before English ones.<\/span><\/p>

                Correct or valid<\/span><\/strong><\/p>

                Part of the standard normalization process involves a correction\/validation. You can use this information to set priorities with your duplicates. By selecting this option, when duplicates are found, valid addresses will receive a higher priority. When you delete your duplicates, a valid record would be retained while others will be deleted.<\/span><\/p>

                Random<\/span><\/strong><\/p>

                If two or more entries are exact matches, this option randomly selects one over the others. This option is always the \u201clast resort\u201d, that is, if other priorities have been selected and there are still duplicates remaining, the random selection will happen.<\/span><\/p>

                Note:<\/span><\/strong><\/p>

                The drop-down list beside the Set Priorities shows the priorities you have selected and the order in which they will be applied to your job.<\/span><\/p>

                \u00a0Superior record creation<\/span><\/strong><\/p>

                This option allows you to create one record using parts of two or more duplicate records. Once you have processed your duplicates, you can access this feature by clicking the \u201cView\/Print\u201d button and selecting \u201cMerge\u201d from the View Database screen. You can then select components from each duplicate record to create a single entry in your database.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t

                \n\t\t\t
                \n\t\t\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

                Priority Duplicate Extraction Duplicate extraction utility allows you to identify and selectively remove duplicate records from your database. If you merge 2 or more files, you can identify records that are common between them. \u00a0This is useful for removing suppression lists, also known as Kill Files. From the Address Management menu, select \u201cPriority Duplicate Extraction\u201d. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"elementor_header_footer","meta":{"site-sidebar-layout":"no-sidebar","site-content-layout":"page-builder","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"disabled","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-1937","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/pages\/1937","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/comments?post=1937"}],"version-history":[{"count":5,"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/pages\/1937\/revisions"}],"predecessor-version":[{"id":5279,"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/pages\/1937\/revisions\/5279"}],"wp:attachment":[{"href":"https:\/\/www.flagshipsoftware.com\/fsweb\/wp-json\/wp\/v2\/media?parent=1937"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}