[sql]UPDATE table_name SET <timestamp col>=<timestamp> + INTERVAL <expr> <unit> WHERE…;[/sql]
We can use the INTERVAL keyword to achieve the same in MySQL.
For example:
[sql]UPDATE table_name SET modified_at=modified_at + INTERVAL -1 DAY WHERE DATE(modified_at)=DATE(NOW());[/sql]
MySQL defines standard formats for expr and unit as illustrated in the following table:
| unit | expr |
|---|---|
| DAY | DAYS |
| DAY_HOUR | ‘DAYS HOURS’ |
| DAY_MICROSECOND | ‘DAYS HOURS:MINUTES:SECONDS.MICROSECONDS’ |
| DAY_MINUTE | ‘DAYS HOURS:MINUTES’ |
| DAY_SECOND | ‘DAYS HOURS:MINUTES:SECONDS’ |
| HOUR | HOURS |
| HOUR_MICROSECOND | ‘HOURS:MINUTES:SECONDS.MICROSECONDS’ |
| HOUR_MINUTE | ‘HOURS:MINUTES’ |
| HOUR_SECOND | ‘HOURS:MINUTES:SECONDS’ |
| MICROSECOND | MICROSECONDS |
| MINUTE | MINUTES |
| MINUTE_MICROSECOND | ‘MINUTES:SECONDS.MICROSECONDS’ |
| MINUTE_SECOND | ‘MINUTES:SECONDS’ |
| MONTH | MONTHS |
| QUARTER | QUARTERS |
| SECOND | SECONDS |
| SECOND_MICROSECOND | ‘SECONDS.MICROSECONDS’ |
| WEEK | WEEKS |
| YEAR | YEARS |
| YEAR_MONTH | ‘YEARS-MONTHS’ |
If you are trying to read a CSV file in Python 3.6 using this:
[python]
with open(filepath) as f:
reader = csv.reader(f, delimiter=’,’, quotechar=’"’, skipinitialspace=True)
header = next(reader)
# rest of the code
f.close()
[/python]
Then you might get the following (similar)error:
[shell]
Traceback (most recent call last):
File "<filepath>", line 17, in csv_to_json
header = reader.next()
AttributeError: ‘_csv.reader’ object has no attribute ‘next’
[/shell]
This is very simple error to solve.
[python]reader.next()[/python]
is actually Python 2.x syntax and that’s why it doesn’t work in Python 3.x. To make it work in Python 3.x, you should do:
[python]next(reader)[/python]
That’s it. Please comment if you face any issues.
Migrating data from MongoDB (or other data sources like Postgres/Redis/RethinkDB/File) to Elasticsearch might be a painful task. The Elasticsearch is being mainly used for text based search engines and similar niche use-cases in most of its deployments. Very few consider Elasticsearch to be used as a primary datastore(which is not recommended) or for storing some metadata. However, Elasticsearch is more than suitable for such use cases.
And once you decide to use Elasticsearch to replace your existing datastore, you will come to know that there is not much ecosystem/tools available to do that. However, there are indeed some open source tools using which you can achieve the task.
GitHub link: https://github.com/compose/transporter
Quote from Transporter GitHub page – “Transporter allows the user to configure a number of data adaptors as sources or sinks. These can be databases, files or other resources. Data is read from the sources, converted into a message format, and then send down to the sink where the message is converted into a writable format for its destination. The user can also create data transformations in JavaScript which can sit between the source and sink and manipulate or filter the message flow.
Adaptors may be able to track changes as they happen in source data. This “tail” capability allows a Transporter to stay running and keep the sinks in sync.”
I have personally used this tool and it works flawlessly.
[javascript]var source = mongodb({
"uri": "mongodb://127.0.0.1:27017/test"
// "timeout": "30s",
// "tail": false,
// "ssl": false,
// "cacerts": ["/path/to/cert.pem"],
// "wc": 1,
// "fsync": false,
// "bulk": false,
// "collection_filters": "{}",
// "read_preference": "Primary"
})
var sink = elasticsearch({
"uri": "https://localhost:9200/test_index"
// "timeout": "10s", // defaults to 30s
// "aws_access_key": "ABCDEF", // used for signing requests to AWS Elasticsearch service
// "aws_access_secret": "ABCDEF" // used for signing requests to AWS Elasticsearch service
})
t.Source(source).Save(sink)
// t.Source("source", source).Save("sink", sink)
// t.Source("source", source, "namespace").Save("sink", sink, "namespace")[/javascript]
[shell]$ ./transporter run [-log.level "info"] pipeline.js[/shell]
For other use-cases, please explore Transporter documentation on its GitHub page.
While compiling/building a project in Maven, you might sometimes encounter the following error:
[shell]java.lang.OutOfMemoryError: PermGen[/shell]
It might be caused due to any of the following points:
It is a simple fix:
It is very simple to do, just run the following command from the user in which cron needs to be created or add the command to your bash script and run the script from the user in which cron needs to be created:
Shell command:
[shell]
(crontab -l 2>/dev/null; echo "* * * * * /path/to/script or command here") | crontab –
[/shell]
If you need to add it to the root‘s crontab, then run the command after logging in as root, or run the above command with sudo or add the command to your bash script and run the script as sudo or from root user:
[shell]
(sudo crontab -l 2>/dev/null; echo "* * * * * /path/to/script or command here") | sudo crontab –
[/shell]
Although, executing above might give an error message – “no crontab for user”. However, you can ignore the error as it creates the crontab entry anyway.
