How to create a /robots.txt file

How to create a /robots.txt file

Where to put it

The short answer: in the top-level directory of your web server.

The longer answer:

When a robot looks for the "/robots.txt" file for URL, it strips the path component from the URL (everything from the first single slash), and puts "/robots.txt" in its place.

For example, for "http://www.example.com/shop/index.html, it will remove the "/shop/index.html", and replace it with "/robots.txt", and will end up with "http://www.example.com/robots.txt".

So, as a web site owner you need to put it in the right place on your web server for that resulting URL to work. Usually that is the same place where you put your web site's main "index.html" welcome page. Where exactly that is, and how to put the file there, depends on your web server software.

Remember to use all lower case for the filename: "robots.txt", not "Robots.TXT.

See also:

• What program should I use to create /robots.txt?
• How do I use /robots.txt on a virtual host?
• How do I use /robots.txt on a shared host?

What to put in it

The "/robots.txt" file is a text file, with one or more records. Usually contains a single record looking like this:

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~joe/

In this example, three directories are excluded.

Note that you need a separate "Disallow" line for every URL prefix you want to exclude -- you cannot say "Disallow: /cgi-bin/ /tmp/" on a single line. Also, you may not have blank lines in a record, as they are used to delimit multiple records.

Note also that globbing and regular expression are not supported in either the User-agent or Disallow lines. The '*' in the User-agent field is a special value meaning "any robot". Specifically, you cannot have lines like "User-agent: *bot*", "Disallow: /tmp/*" or "Disallow: *.gif".

What you want to exclude depends on your server. Everything not explicitly disallowed is considered fair game to retrieve. Here follow some examples:

To exclude all robots from the entire server

User-agent: *
Disallow: /

To allow all robots complete access

User-agent: *
Disallow:

(or just create an empty "/robots.txt" file, or don't use one at all)

To exclude all robots from part of the server

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/

To exclude a single robot

User-agent: BadBot
Disallow: /

To allow a single robot

User-agent: Google
Disallow:

User-agent: *
Disallow: /

To exclude all files except one

This is currently a bit awkward, as there is no "Allow" field. The easy way is to put all files to be disallowed into a separate directory, say "stuff", and leave the one file in the level above this directory:

User-agent: *
Disallow: /~joe/stuff/

Alternatively you can explicitly disallow all disallowed pages:

User-agent: *
Disallow: /~joe/junk.html
Disallow: /~joe/foo.html
Disallow: /~joe/bar.html

htaccess trick

1) Force www to non www or v.v
#Force www:
RewriteEngine on
RewriteCond %{HTTP_HOST} ^example.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301,NC]

#Force non-www:
RewriteEngine on
RewriteCond %{HTTP_HOST} ^www\.example\.com [NC]
RewriteRule ^(.*)$ http://example.com/$1 [L,R=301]

2) Exclude folder for rewrite

RewriteRule ^(admin|user)($|/) - [L]




How to fix “Cannot send headers; headers already sent”

Warning: Cannot modify header information - headers already sent by (output started at /some/file.php:12) in /some/file.php on line 23

Solution: Add or Enable: output_buffering = On; in php.ini

Date Range Jquery and Disable past date

<code>
var dateToday = new Date();
var dates = $("#from, #to").datepicker({
    defaultDate: "+1w",
    changeMonth: true,
    numberOfMonths: 3,
    minDate: dateToday,
    onSelect: function(selectedDate) {
        var option = this.id == "from" ? "minDate" : "maxDate",
            instance = $(this).data("datepicker"),
            date = $.datepicker.parseDate(instance.settings.dateFormat || $.datepicker._defaults.dateFormat, selectedDate, instance.settings);
        dates.not(this).datepicker("option", option, date);
    }
});

Demo

Disable submit button until all field selected

<code>
$('#submitForm').prop('disabled', true);;

function updateFormEnabled() {
    if (verifyAdSettings()) {
        //$('#submitForm').attr('disabled', '');
        $('#submitForm').prop('disabled', false);
    } else {
        //$('#submitForm').attr('disabled', 'disabled');
        $('#submitForm').prop('disabled', true);
    }
}

function verifyAdSettings() {
    if ($('#cities1').val() != '' && $('#cities2').val() != '' && $('#start').val() != '') {
        return true;
    } else {
        return false
    }
}

$('#cities1').change(updateFormEnabled);
$('#cities2').change(updateFormEnabled);
$('#start').change(updateFormEnabled);

Required Jquery 2.1

Demo

jQuery match up two different select boxes

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

<select class="state1" id="state1" name="state1">
  <option value="AL">Alabama</option>
  <option value="AK">Alaska</option>
  <option value="AZ">Arizona</option>
  <option value="AR">Arkansas</option>
  <option value="CA">California</option>
  <option value="CO">Colorado</option>
  <option value="CT">Connecticut</option>
</select>

<select id="state2" name="state2" size="0" class="state2">    
  <option value="55" class="has-children">AL</option>
  <option value="25" class="has-children">AK</option>
  <option value="200" class="has-children">AZ</option>
  <option value="123" class="has-children">AR</option>
  <option value="216" class="has-children">CA</option>
  <option value="275" class="has-children">CO</option>
  <option value="340" class="has-children">CT</option>
</select>

$(document).ready(function () {   
 $("#state1").change(function() {
  var val = $(this).val();
  $('#state2 option').each(function(){
   var val2 = $(this).html();
   console.log(val2);
   if (val2 == val) {
    //$(this).prop('selected', true);
    $(this).remove(); // remove option from select
   } else {
    $(this).prop('selected', false);
   }
  });
 });
 

});
</script>

Web Plugin

1- Select2 https://select2.github.io/examples.html
2- Selectize https://selectize.github.io/selectize.js/

Bootstrap Datetimepicker:  https://eonasdan.github.io/bootstrap-datetimepicker/

Bootstrap Date Range Picker: http://www.daterangepicker.com/

Dynamically populate a select element from JSON data with jQuery/Ajax

<html>

  <head>

    <script type="text/JavaScript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js"></script>

  </head>

  <body>

    <select id="people"></select>

  </body>

The JSON file we’ll be using consists of a collection of ‘person’ data in a valid JSON form, such as:

{"person": [

  {

    "id": "1",

    "name": "Person1"

  },

  {

    "id": "2",

    "name": "Person2"

  },

  {

    "id": "3",

    "name": "Person3"

  }

]}




  <script type="text/JavaScript">

//getJSON

    //get a reference to the select element

    var $select = $('#people');

    //request the JSON data and parse into the select element

    $.getJSON('person.JSON', function(data){

      //clear the current content of the select

      $select.html('');

      //iterate over the data and append a select option

      $.each(data.person, function(key, val){

        $select.append('<option id="' + val.id + '">' + val.name + '</option>');

      })

    });

//AJAX

//get a reference to the select element

$select = $('#people');

//request the JSON data and parse into the select element

$.ajax({

  url: 'person.JSON',

  dataType:'JSON',

  success:function(data){

    //clear the current content of the select

    $select.html('');

    //iterate over the data and append a select option

    $.each(data.person, function(key, val){

      $select.append('<option id="' + val.id + '">' + val.name + '</option>');

    })

  },

  error:function(){

    //if there is an error append a 'none available' option

    $select.html('<option id="-1">none available</option>');

  }

});

  </script>

</html>

https://developerdan77.wordpress.com/2011/10/14/dynamically-populate-a-select-element-from-json-data-with-jquery/